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Foreword 

For  many  years,  the  U.S.  Army  recognized  the  need  to  share  information 
or  lessons  gained  from  training  and  actual  combat  operations.  During 
World  War  II  and  the  Korean  War,  the  Army  published  “combat  bulletins” 
in  an  attempt  to  share  combat  experiences  with  other  Soldiers.  During  the 
Vietnam  War,  Army  units  published  quarterly  operational  reports  that  made 
an  effort  to  share  lessons  from  combat  operations.  By  doing  this,  units 
learned  from  the  mistakes  others  made  and  were  given  an  opportunity  to 
avoid  the  same  problems. 

Although  these  procedures  were  successful,  the  Army  did  not  have  a  formal 
or  permanent  program  in  place  to  collect,  analyze,  and  share  lessons  in 
both  peacetime  and  wartime.  As  a  result,  the  Army  established  the  Center 
for  Army  Lessons  Learned  (CALL)  in  1985.  The  Army  had  officially 
recognized  the  need  to  share  lessons  gained  from  training  experiences  and 
actual  combat  operations.  Since  the  inception  of  CALL,  the  Army,  as  a 
learning  organization,  greatly  evolved  over  time.  CALL  also  evolved. 

Now,  military  communities,  civilian  governmental  agencies,  and  the 
corporate  world  recognize  the  importance  of  sharing  knowledge  and 
learning  from  past  experiences.  In  many  ways,  the  U.S  Army  led  this  effort 
by  allowing  commanders  to  make  honest  mistakes  in  training,  talk  about 
those  mistakes  openly,  and  share  what  was  done  to  correct  those  mistakes 
with  other  units  about  to  undergo  the  same  training  experience.  This  is  an 
essential  precondition  for  having  an  effective  lessons  learned  (LL)  program: 
the  ability  to  self-analyze  and  self-criticize  in  an  atmosphere  where  there  is 
no  blame.  The  results  were  instrumental  in  changing  the  Army  and  creating 
a  learning  environment  that  won  the  Cold  War,  ensured  victory  in  Desert 
Storm,  and  continues  to  support  our  Soldiers  in  Iraq  and  Afghanistan  today. 

With  that  said,  the  intent  of  this  handbook  is  not  to  explain  how  CALL 
does  business.  Rather,  it  takes  a  holistic  look  at  what  CALL  does  today  and 
combines  that  with  numerous,  successful  LL  programs  to  lay  a  foundation 
for  a  “generic”  LL  capability  that  can  be  used  as  a  “menu”  of  options  to 
develop  your  own  LL  program.  There  are  no  hard-and-fast  rules  for  how 
to  set  up  an  LL  program.  Typically,  any  LL  effort  will  depend  on  what 
the  organization  is  trying  to  achieve  and  its  level  of  resourcing.  Every 
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organization  must  develop  its  own  way  of  working  with  observations, 
insights,  lessons,  and  best  practices  that  suits  the  nature  of  the  knowledge  it 
requires  and  its  organizational  culture. 

Our  hope  is  this  handbook  will  assist  you  in  either  refining  your  LL 
capability  or  establishing  a  new  organizational  goal  toward  LL.  We  wish 
you  the  best  of  luck  in  that  endeavor. 


THOMAS  H.  ROE  ' 

COL.  IN 

Director,  Center  for  Army  Lessons  Learned 
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Introduction 

This  handbook  provides  leaders  and  members  of  an  organization  with 
a  “how-to”  guide  for  establishing  a  lessons  learned  (LL)  capability.  It 
promises  to  detail  what  the  LL  process  is  and  how  to  apply  all  the  available 
tools  to  establish  your  own  LL  program. 

Different  organizations  in  the  LL  community  vary  terms  that  are  not 
necessarily  consistent.  This  handbook  attempts  to  simplify  and  explain  these 
terms  to  satisfy  the  development  of  a  generic  LL  capability.  In  developing 
this  LL  handbook,  the  Center  for  Army  Lessons  Learned  (CALL)  drew 
from  the  experience  and  techniques  used  by  the  Army,  Marines,  Air  Force, 
NATO,  and  other  U.S.  government  agencies.  It  examined  the  processes  in 
place  to  collect,  analyze,  disseminate,  and  archive  observations,  insights, 
lessons,  and  best  practices  and  distilled  them  down  to  a  simplified  list  of 
functions  any  organization  could  choose  from  to  establish  an  effective 
program  that  fits  its  level  of  resourcing.  The  goal  of  this  handbook  is  to 
provide  a  one-source  document  that  anyone  can  use  to  understand  how  a 
basic  program  is  achieved  to  improve  organizational  effectiveness. 

All  organizations  must  learn  to  adapt  and  change.  An  enabler  to  do  this  is 
understanding  the  value  of  a  lesson  learned.  The  military  has  been  a  leader 
in  understanding  the  vital  importance  of  an  LL  capability.  LL  processes 
have  been  used  in  commercial,  government,  and  military  organizations 
since  the  late  1980s  to  capture,  store,  and  share  experiential  working 
knowledge. 

Although  this  is  a  CALL  product,  it  is  not  a  primer  on  how  CALL  does 
business.  We  believe  this  handbook  will  have  applicability  for  military, 
government,  and  civilian  organizations  that  want  to  develop  an  LL 
capability.  Although  it  does  use  some  military  examples,  we  will  try  to 
demystify  military  terminology  and  keep  the  explanations  as  basic  and 
simple  as  possible. 

There  are  three  chapters  in  this  handbook,  followed  by  a  series  of 
appendices.  Chapter  1  briefly  discusses  what  an  LL  program  is  and  why 
you  should  have  one.  Chapter  2  outlines  the  functions  of  an  LL  program. 
Chapter  3  discusses  the  organizational  considerations  involved  when 
establishing  a  program.  In  the  appendices,  you  will  find  examples  of  how 
other  LL  centers  operate  their  programs  and  supplemental  information  on 
after  action  reviews/reports,  collection  plans,  and  interviews. 
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Some  organizations  that  have  developed  LL  programs: 


Government 

Military 

Nonmilitary 

U.S. 

U.S. 

Air  Force 

Army 

Coast  Guard 

Joint  forces 

Marine  Corps 

Navy 

Department  of  Energy 

Society  for  Effective  Lessons  Learned  Sharing 
NASA  (Ames,  Goddard) 

International 

International 

Canadian  Army 
Lessons  Learned 
Centre 

European  Space  Agency 

Italian  (Alenia) 

French  (CNES) 

Japanese  (NASDA) 

United  Nations 

'Those  who  cannot  remember  the  past  are  doomed  to  repeat  it.” 

—  George  Santayana,  Philosopher 
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Chapter  1 

Why  a  Lessons  Learned  Program? 

Why  does  an  organization  need  a  lessons  learned  (LL)  capability?  Before 
we  discuss  that,  it  is  important  to  understand  what  is  a  “lesson”  and  what 
is  a  “lesson  learned.”  A  lesson  is  knowledge  or  understanding  gained 
by  experience.  The  experience  may  be  positive  (a  best  practice),  as  in  a 
successful  test,  mission,  exercise,  or  workshop,  or  negative,  as  in  a  mishap 
or  failure.  Successes  and  failures  are  both  considered  sources  of  lessons. 

A  lesson  must  be  significant  in  that  it  has  a  real  or  assumed  impact  on 
everyday  operations.  It  must  be  valid  in  that  it  is  factually  and  technically 
correct;  applicable  in  that  it  identifies  a  specific  design,  process,  or 
decision;  and  it  reduces  or  eliminates  the  potential  for  failures  and  mishaps 
or  reinforces  a  positive  result.  Basically,  it  is  the  knowledge  acquired 
from  an  observation  or  an  adverse  experience  that  causes  a  worker  or  an 
organization  to  improve. 

A  lesson  is  an  LL  when  you  can  measure  a  change  in  behavior.  Obviously, 
this  change  in  behavior  needs  to  be  of  a  positive  nature  that  improves 
performance.  The  U.S.  Army,  with  over  25  years  of  focused  LL  experience 
and  one  of  the  world’s  leaders  in  experiential  learning,  still  struggles  with 
actually  “learning”  lessons  once  identified.  Even  though  there  are  many 
understandable  reasons  for  this,  you  cannot  give  up.  Other  organizations 
complain  that  once  you  identify  a  lesson,  it  ends  up  in  some  database 
and  you  quickly  forget  it.  The  irritation  of  every  LL  specialist  is  seeing 
important  lessons  collected  and  never  being  shared  or  resolved.  This  takes 
time  and  effort  and,  in  most  instances,  money.  Often  there  is  no  obvious 
“owner”  of  the  lesson  identified,  and  there  is  rarely  a  system  set  up  to 
resolve  the  issue  and  implement  corrective  actions. 

Do  not  be  discouraged.  There  are  some  very  sound  reasons  why  your 
organization  needs  an  LL  capability  that  can  evolve  into  an  effective 
program.  Here  are  just  a  few: 

•  It  saves  time  by  providing  a  central  location  for  efficient  searches  of 
valuable  LL  information. 

•  It  helps  reduce  or  avoid  costs  by  providing  information  on  success 
stories  that  you  may  be  able  to  implement  or  mistakes  that  you  may  be 
able  to  avoid. 

•  You  can  expand  your  information  network  by  providing  information¬ 
sharing  opportunities  by  connecting  with  other  sites,  “experts,”  or 
people  doing  similar  work. 
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•  Most  importantly,  it  can  reduce  the  risk  of  repeated  mistakes  and 
improve  the  chance  that  success  is  continual. 

Once  you  have  decided  that  an  LL  program  is  what  you  want  to  implement, 
consider  the  following  points: 

•  Secure  executive  and  leadership  “buy-in.” 

•  Avoid  the  “evaluation  or  inspection”  tag. 

•  Push  best  practices  out. 

•  Start  small,  then  grow  slowly. 

•  Show  that  the  process  can  bring  change. 

These  are  also  some  advantages  to  capturing  project  lessons  and  turning 
them  into  knowledge: 

•  To  improve  project  management  processes. 

•  To  improve  management  decision  making  (develop  new  strategies). 

•  To  improve  personnel  performance. 

•  To  increase  organizational  knowledge. 

•  To  save  lives  and  resources  (money,  supplies,  time). 

“[  It  is  estimated]  that  Fortune  500  companies  lose  $31.5  billion  each 
year  because  they  don’t  share  knowledge.” 

—  “20/20  Foresight,”  PM  Network,  September  2004 


As  mentioned  in  the  foreword  of  this  document,  there  is  one  important 
implication  that  is  critical  to  understand  before  implementing  any  LL 
program.  This  implication  involves  the  “culture”  or  attitude  of  your 
organization.  Is  your  organization  willing  to  openly  discuss  its  mistakes, 
and  is  it  willing  to  share  those  mistakes  across  organizational  lines  to  make 
everyone  better?  If  not,  it  will  be  very  difficult  to  implement  an  effective 
LL  program.  If  you  are  willing  to  share  those  mistakes,  can  you  do  so  in 
an  atmosphere  that  avoids  direct  blame  on  those  willing  to  bring  problems 
forward?  You  must  be  able  to  do  this  to  be  a  learning  organization  that 
facilitates  knowledge  sharing.  Within  some  cultures,  this  is  very  difficult 
to  do.  The  act  of  “saving  face”  precludes  individuals  from  admitting  their 
mistakes.  However,  some  armies  have  successfully  overcome  this  cultural 
difference  after  working  with  the  U.S.  military  and  understanding  the 
importance  of  learning  from  their  past.  Again,  being  able  to  self-examine 
and  self-criticize  in  an  atmosphere  where  everyone  can  avoid  blame  is 
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essential  for  honest  and  open  discussion.  This  is  an  essential  precondition 
for  an  effective  LL  program. 

The  LL  program  can  be  a  linking  mechanism  that  connects  existing  LL 
initiatives,  a  series  of  tools  that  facilitate  learning  and  information  transfer, 
and  a  broad  network  of  individuals  who  contribute  to  the  sharing  process. 

It  has  the  potential  to  be  a  multifaceted  initiative  that  uses  information 
technologies  to  link  LL  programs,  rapidly  transfer  time-critical  LL 
information  to  key  points  of  contact,  and  provide  access  to  pertinent 
information  available  outside  of  the  organization. 

Every  project  your  agency  undertakes  demands  substantial  concentration 
and  focus,  creativity  in  problem  solving,  and  a  wide  range  of  practical 
skills.  However,  unless  your  agency  makes  a  specific  effort  to  retain  the 
knowledge  and  experience  learned  on  one  project,  much  of  the  value 
will  be  lost.  Establishing  an  LL  process  will  help  your  agency  leam  from 
its  mistakes  and  apply  new  knowledge  and  experience  to  other  projects. 
Activities  are  a  routine  part  of  life,  allowing  us  to  build  up  and  share  our 
experience  as  human  beings,  thus  leading  us  to  LL. 

In  the  military  context,  LL  are  a  natural  product  of  operations,  training, 
and  exercises  and,  indeed,  any  routine  work.  During  the  course  of  our 
activities,  most  of  us  will  recognize  easier,  more  efficient  ways  of  doing 
things,  problems  that  are  avoidable,  or  issues  we  can  prevent  our  colleagues 
and  successors  from  suffering  from.  The  LL  process  is  simply  the  process 
of  trying  to  ensure  that  you  repeat  good  practices  or  that  the  same  problem 
does  not  occur  again. 

Although  observations  are  the  first  building  blocks  of  the  process  and  initial 
identification  of  a  good  or  bad  practice  or  event,  an  organization  generally 
has  topics  it  wants  to  focus  on  or  issues  it  has  determined  are  problematic. 
Each  of  these  issues  requires  a  determined  method  to  collect  information 
that  can  be  subject  to  further  analysis.  Analysis  results  in  the  development 
of  lessons,  best  practices,  and  recommendations  for  corrective  actions  to 
help  resolve  the  issue.  To  learn  a  lesson,  you  will  need  recommendations  to 
lead  you  to  the  work  that  will  help  resolve  and  implement  the  lesson.  Figure 
1-1  provides  the  terms  and  definitions  that  will  be  used  throughout  this 
handbook. 
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ISSUB 


Observation 


Issue:  A  general  topic  area  that  is  identified  as  a 
potential  problem  for  a  collection  effort.  Example. 
Operations  Iri  Hot  Climates 

Observation.  A  statement  of  the  conditions 
experienced  or  observed,  Example-  Extreme  heat  is 
causing  vehicle  engines  to  overheat 

lesson.  Potential  solutions  to  the  problems 
experienced  as  a  result  of  the  observation  (noumilly 
results  in  a  recommendation)  Example-  Adding  an 
oversized  radiator  to  vehicle  engines  reduces 
overbearing  problems. 

Recommendation.  A  proposed  corrective  action  that 
when  Implemented,  should  correct  the  problem 
Example:  All  vehicles  '.tumid  be  fitted  with  oversized 
radiators. 

Best  Practice.  A  solution  to  a  problem  that  Is  already 
implemented  and  working,  txample;  vehicle 
overheating  problems  were  eliminated  bv  adding 
oversized  radiators 

lesson  learned-  An  implemented  corrective  action 
that  leads  to  improved  performance  or  an  observed 
change  in  behavior  Example  Once  oversized 
radiators  were  installed  on  vehicles,  they  no  longer 
experienced  overheating. 


Figure  1-1.  Definitions 

Following  is  a  brief  explanation  of  the  functions  that  are  important  to  any 
viable  LL  program.  These  functions  will  be  explained  in  greater  detail  in 
Chapter  2. 

Collect 

You  must  first  have  the  ability  to  collect  information  on  specific  topics 
or  issues  of  interest.  There  are  many  ways  to  do  this.  Information  can 
be  “pulled”  into  the  process  through  direct  collection  efforts  or  it  can  be 
“pushed”  into  the  process  by  organizations,  units,  and  individuals  from  the 
bottom  lip.  This  information  is  usually  unrefined  observations  requiring 
additional  analysis.  You  can  accomplish  collection  through  direct  and 
indirect  methods  and  run  a  full  range  of  processes  from  answering  data 
calls  in  support  of  a  request  for  information,  to  formal  collections  involving 
well-developed  questionnaires,  to  a  team  of  experts  brought  together  to 
specifically  gather  further  information  for  analysis.  Different  techniques 
and  tools  for  collection  include  direct  observation,  in-person  interviews, 
surveys,  database  submissions  by  participants,  data  and  database  mining, 
and  document  capturing. 
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A  good  way  to  accelerate  an  LL  collection  effort  is  to  motivate  individual 
and  organizational  behavior  using  a  “market  strategy.”  A  market  strategy 
treats  the  exchange  of  information  as  a  tangible  transaction  between 
someone  who  has  information  and  someone  who  needs  the  information. 

In  the  commercial  world,  this  equates  to  linking  a  buyer  and  a  seller  with 
currency  exchange  based  on  the  value  of  the  product  or  service.  In  an 
information-exchange  market,  the  dynamic  is  the  same,  but  the  “currency” 
involved  is  usually  not  cash. 

Analyze 

The  transformation  of  an  observation  to  lesson  identified  occurs  when  the 
analysis  reveals  the  root  causes  of  the  problem  and  identifies  the  appropriate 
remedial  action  or  corrective  action.  Analysis  typically  answers  the  “who, 
what,  when,  where,  and  why”  to  identify  the  root  causes  of  the  problem  or 
success.  Analysis  results  in  lessons  that  lead  to  recommendations,  which 
ultimately  enhance  organization  performance  once  implemented.  Thorough 
analysis,  when  done  properly,  is  one  of  the  most  difficult  and  time- 
consuming  aspects  of  the  LL  process. 

Share 

The  sharing  of  lessons  in  and  between  organizations  ensures  everyone 
benefits  from  the  knowledge  gained.  Lessons  are  shared  through  many 
venues:  briefings,  bulletins,  reports,  e-mails,  websites,  database  entries, 
etc.  Sharing  lessons  and  making  them  available  to  everyone  should  be  the 
primary  goal  of  a  good  LL  program.  It  can  potentially  reduce  risk,  improve 
efficiency,  and  enhance  the  cost  effectiveness  of  processes  and  operations. 
Sharing  of  data  between  LL  professionals,  historians,  and  other  learning 
organizations  is  also  encouraged.  The  guiding  principle  in  executing  a 
sharing  strategy  is  to  get  the  right  information  to  the  right  person  at  the 
right  time.  While  conducting  shared  activities  using  different  approaches,  it 
is  imperative  to  make  sure  we  disseminate  accurate  information.  The  next 
and  sometimes  more  difficult  challenge  is  to  have  a  means  to  rapidly  get 
the  information  to  those  who  need  it  the  most  to  leverage  the  power  of  this 
information  by  reducing  organizational  obstruction  to  the  dissemination  of 
knowledge. 

Archive 

The  archive  function  is  a  broad  term  that  encompasses  several  required 
capabilities  that  most  LL  programs  desire.  The  ability  to  archive  information 
and  manage  records,  both  print  and  electronic,  allows  access  to  them  at  a 
future  date.  To  accomplish  this  task,  develop  digital  repositories  (called 
digital  libraries  or  archives)  to  store  information,  facilitate  the  historical 
preservation  of  information,  and  allow  users  to  conduct  research.  It  shoidd 
provide  a  logical  system  for  organizing  information  that  is  easily  retrievable 
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and  provided  to  any  requestor.  Powerful  search  engines  are  required  that 
permit  rapid,  user-friendly  searches.  If  required,  repositories  should  have 
the  capability  to  store  and  guard  classified,  sensitive  or  proprietary,  and 
unclassified  data.  Understand  that  the  archive  function  needs  to  become  and 
remain  an  ongoing  process  or  the  repository  will  soon  die  of  its  own  weight 
and  growing  irrelevance. 

Resolve 

The  most  challenging  component  of  any  LL  program  is  establishing  a 
process  to  legitimately  resolve  issues  once  analysis  is  completed  and  as 
early  and  quickly  in  the  process  as  possible.  Whether  or  not  the  LL  program 
has  responsibility  for  the  issues-resolution  process,  the  organization’s  ability 
to  change  behavior  by  implementing  a  lesson  is  ineffective  unless  that 
change  can  be  observed  and  a  determination  made  that  the  lesson  is  learned. 
In  other  words,  the  corrective  actions  have  enhanced  performance.  To  do 
this  requires  a  deliberate  process  to  commit  resources,  make  decisions, 
implement  those  decisions,  and  observe  the  results.  If  the  process  is 
cumbersome  and  too  difficult,  expect  the  results  to  be  less  than  optimal.  The 
first  objective  is  to  handle  the  corrective  action  at  the  lowest  level  possible. 
The  issues  that  rise  to  the  next  level  of  attention  are  those  that  units  or 
organizations  are  unable  to  correct  internally.  They  are  issues  that  require 
outside  assistance  to  correct  or  assistance  from  external  sources. 

Assess 

There  are  several  ways  to  determine  if  the  LL  program  is  effective.  Any  LL 
program  evaluates  the  expenditure  of  resources  against  the  desired  results. 
This  is  difficult  to  determine  through  quantitative  analysis.  However,  some 
ways  to  evaluate  the  effectiveness  of  an  LL  program  may  validate  and 
justify  its  continued  resourcing.  Assessment  of  LL  effectiveness  can  be 
broken  into  several  components:  organization  behavior,  organization  or  unit 
performance,  and  mission  effectiveness.  Each  of  these  techniques  are  ways 
to  determine  if  your  program  is  effective. 

By  reading  this  handbook,  you  have  taken  the  right  step  in  establishing  an 
LL  program.  Additional  LL  references  and  contact  information  are  located 
in  Appendix  D  should  you  need  further  assistance. 
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Chapter  2 

Functions  of  a  Lessons  Learned  Capability 

Lessons  learned  (LL)  programs  will  vary  according  to  the  needs  of  the 
organization;  however,  most  will  have  at  least  six  functions  as  shown  in 
Figure  2-1.  Each  of  these  functions  is  explained  in  greater  detail  in  this 
chapter. 


Collect 

Analyze 

Share 

Archive 

Resolve 

Assess 

Figure  2-1.  LL  functions 


Collect 

The  first  function  of  any  LL  capability  or  program  is  the  collection  of 
information  relevant  to  an  issue  that  someone  has  determined  requires 
analysis.  In  an  LL  program,  the  issues  are  usually  topics  the  organization 
realizes  are  causing  concern  or  problems,  and  the  organization  wants  to 
determine  a  better  way  to  do  business  or  enhance  performance.  Historically, 
most  organizations  are  passive  when  it  comes  to  reporting  problems  and 
potential  solutions  shared  with  other  like  organizations  so  those  other 
organizations  do  not  encounter  the  same  difficulties.  Normally,  if  someone 
does  not  make  the  effort  to  “pull”  these  issues  from  the  organization,  do 
not  expect  them  to  “push”  them  to  you.  Many  reasons  exist  for  this,  such 
as  operational  pace,  shortages  of  manpower,  time  constraints,  a  lack  of 
understanding  the  importance  of  sharing  information,  or  no  process  in  place 
that  facilitates  the  sharing.  Ultimately,  the  goal  of  any  collection  effort  is  to 
gather  enough  information  to  have  informed  analysis  that  resolves  the  issue 
so  other  organizations  can  benefit  from  the  experiences  of  those  who  have 
gone  before  them. 

We  do  not  collect  lessons  to  evaluate  an  organization;  rather,  we  collect 
lessons  to  help  organizations  improve.  If  perception  of  the  LL  program 
is  as  an  evaluator,  an  inspector,  or  an  agency  of  internal  review,  no  one 
will  be  willing  to  share  problems  and  corrective  actions.  A  policy  of 
nonattribution  may  be  appropriate.  Make  every  effort  to  avoid  personally 
naming  individuals;  instead,  use  duty  positions  or  work  positions.  Try  to 
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avoid  naming  specific  units  or  offices  if  possible.  For  example,  if  you  were 
collecting  on  the  1st  Battalion,  502nd  Aviation  Regiment,  instead  of  saying 
1-502  Aviation,  simply  say  an  aviation  battalion.  In  other  words,  be  generic 
when  it  comes  to  sources  but  specific  when  it  comes  to  issues.  To  get  the 
desired  access  and  have  credibility  with  your  organizations,  the  LL  program 
must  be  seen  as  trustworthy,  unbiased,  and  able  to  guard  sources  when 
asked  to  do  so.  That  said,  information  gathered  from  a  collection  should  not 
be  watered  down  or  altered.  Collect  the  facts  and  only  the  facts,  and  try  to 
avoid  opinions. 

Finally,  there  is  a  tendency  in  any  LL  program  to  focus  on  what  is  not 
working  well  (the  negative).  A  good  LL  program  should  also  collect 
information  on  what  is  working  well  (the  positive).  “Best  practices”  are 
what  we  call  positive  observations.  Certainly,  the  tendency  is  to  focus  on  the 
problem  areas,  but  you  may  find  organizations  that  have  mastered  a  certain 
task  and  now  can  share  those  best  practices  with  others.  As  a  rule  of  thumb, 
expect  that  80  percent  of  what  you  collect  may  be  negative  (in  an  attempt  to 
improve);  however,  20  percent  should  be  positive  best  practices. 

Collection  opportunities 

Opportunities  to  collect  information  on  issues  conform  to  the  mission  of 
the  organization  the  LL  program  is  supporting.  For  example,  a  military 
organization  may  have  the  responsibility  to  protect  our  nation  from  foreign 
adversaries  but  also  provide  support  to  civil  operations  in  times  of  national 
emergency.  When  you  analyze  that  mission,  there  are  many  collection 
opportunities  that  exist  that  have  the  ability  to  provide  vital  observations 
and  lessons  on  how  to  best  conduct  those  operations  in  the  future.  Although 
the  participants  should  be  the  first  line  of  observation  reporting,  often  it 
becomes  necessary  for  the  LL  organization  to  be  proactive  in  the  collection 
process.  The  following  is  a  potential  list  of  collection  opportunities: 

•  Training  events  and  exercises  that  prepare  the  unit  for  each  of  its 
specified  missions. 

•  Experiments  and  testing  that  support  the  unit’s  mission. 

•  Planning  sessions  and  conferences  that  support  the  unit’s  mission. 

•  Observing  the  actual  event  or  mission. 

As  you  can  see  from  this  list,  there  are  several  opportunities  to  gather 
information.  To  get  the  maximum  benefit  from  these  opportunities  and 
match  them  to  the  available  resources  requires  a  long-range  collection 
schedule  be  developed  that  overlaps  the  various  opportunities  and  phases 
them  in  over  time  so  the  workload  is  manageable.  Typically,  this  schedule 
runs  for  one  year  and  makes  an  effort  to  lock  in  dates  and  locations  for  each 
collection  effort.  In  the  example  above,  the  occurrence  of  a  natural  disaster 
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is  difficult  if  not  impossible  to  plan  for;  therefore,  the  LL  program  must 
have  the  capability  to  initiate  impromptu  collections  on  very  short  notice. 
Independent  LL  organizations  may  need  a  collection  schedule;  however, 
organizations  with  smaller,  internal  LL  programs  may  not. 

Determining  critical  issues 

One  of  the  greatest  challenges  of  the  collection  phase  is  determining  what 
exactly  are  the  issues  that  require  a  collection  effort  be  analyzed.  There  are 
several  ways  to  accomplish  this.  Often  issues  are  “command  directed.”  A 
higher  headquarters,  agency,  or  organizational  leader  directs  you  to  collect 
on  a  certain  issue  of  interest  to  them.  This  makes  life  simple  and  usually 
allows  you  to  avoid  a  deliberate  process  whereby  the  LL  program  must 
decide  what  issues  are  most  important.  Sometimes  a  “shotgun”  approach  is 
effective.  This  approach  understands  that  in  every  complex  operation  there 
are  always  issues  that  require  work,  and  if  the  LL  program  is  working,  the 
issues  will  certainly  be  discovered.  The  disadvantage  with  this  approach  is 
that  you  do  not  focus  and  you  have  a  tendency  to  look  at  everything,  which 
dilutes  your  efforts. 

Another  method  might  be  the  feedback  gained  from  multiple  units  or 
organizations  that  have  expertise  performing  a  certain  function  that  is  not 
working  properly.  When  numerous  experts  in  the  field  are  all  reporting  the 
same  issue,  it  is  probably  critical.  For  example,  after  the  initial  invasion  of 
Iraq  in  2003,  the  U.S.  Army  experienced  problems  defeating  the  rocket- 
propelled  grenade  that  was  destroying  the  new  Stryker  combat  vehicle. 
Because  of  this  immediate  feedback  from  commanders  in  the  field,  the 
Army  devised  a  defensive  shield  that  could  be  constructed  around  the 
vehicle  to  detonate  the  missiles  prematurely,  rendering  them  ineffective. 

Finally,  a  more  analytical  and  deliberate  approach  would  require  some 
degree  of  analysis  at  home  station  to  detennine  the  issues  that  are  most 
important  to  your  leadership.  A  deliberate  approach  focuses  your  time  and 
resources  but  requires  a  decision  process  to  be  developed  to  determine 
what  is  critical.  One  way  to  do  this  is  to  study  past  operations,  reports, 
articles,  and  briefings  and  begin  to  compile  a  list  of  issues  that  seem  to 
reoccur  and  cause  difficulty.  You  can  weight  each  issue  with  its  degree  of 
importance,  assess  risk,  rank  order  the  issues,  prioritize  resources,  and  make 
a  recommendation  to  senior  leadership  on  which  issues  are  most  important 
for  the  purpose  of  collection.  Each  collection  effort  will  more  than  likely 
uncover  new  issues  that  can  be  added  to  the  existing  list.  Once  again,  you 
will  need  to  go  through  a  process  to  determine  which  issues  are  most  critical 
for  future  collection  efforts. 
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The  collection  plan 

Once  you  have  determined  the  critical  issues,  you  will  need  to  develop  a 
collection  plan  to  guide  the  collection.  A  collection  plan  can  be  as  simple  as 
a  list  of  questions  you  desire  to  ask  the  organization  and  its  subject  matter 
experts  (SMEs)  or  as  detailed  as  the  following: 

•  Specific  guidance  for  the  observers. 

•  A  delineation  of  collection  responsibilities  for  each  observer. 

•  A  list  of  questions  for  each  issue. 

•  A  determination  of  the  collection  methodology  for  asking  each 
question,  which  includes  who  will  be  asked  each  question. 

•  Identification  of  documents  and  resources  to  be  used  in  conjunction 
with  the  collection  effort. 

•  A  schedule  of  interviews. 

•  Travel,  support,  and  information  management. 

However,  the  common  threads  in  all  collection  plans  are  the  questions.  You 
never  answer  a  good  question  with  a  yes  or  no.  The  question  should  be 
worded  in  such  a  way  that  the  person  answering  the  question  must  give  a 
full  account  of  the  process  the  organization  is  using.  Follow  these  examples: 

WRONG:  Is  your  unit  performing  route  clearance  operations? 

Are  we  measuring  performance  improvements  in  the  targeted  areas? 

RIGHT:  How  is  your  unit  performing  route  clearance  operations? 

How  might  we  measure  performance  improvements  in  the  target 

areas? 

As  stated  previously,  you  must  avoid  the  tendency  to  collect  on  too  many 
issues.  More  is  not  necessarily  better.  As  a  general  rule  but  based  on  the 
size  of  your  collection  team,  more  than  10  issues  for  any  one  collection 
effort  gets  cumbersome.  If  a  collection  team  can  deploy  with  six  to  12 
lead  questions  already  prepared  for  each  issue,  that  normally  gives  you  the 
degree  of  fidelity  you  need  to  get  good  results.  Additionally,  the  collection 
plan  is  only  a  plan.  Once  you  begin  to  ask  questions  to  the  interviewees, 
there  is  nothing  wrong  with  developing  new  questions  “on-the-fly”  that 
better  address  the  issues  based  on  the  information  you  are  gaining  from  the 
unit’s  SMEs.  Do  not  get  locked  into  the  plan.  It  is  only  a  start-point  to  get 
you  moving  in  the  right  direction. 
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Finally,  there  is  another  collection  plan  technique  you  may  or  may  not  elect 
to  use.  It  involves  the  use  of  a  hypothesis  statement  for  each  issue.  The 
hypothesis  statement  is  a  statement  that  defines  what  you  want  to  confirm  or 
deny  about  the  issue.  For  example: 

Flypothesis:  Brigade  combat  teams  continue  to  have  difficulty 
clearing  routes  due  to  the  enemy’s  use  of  improvised  explosive 
devices  (IEDs). 

In  this  example,  there  is  probably  some  anecdotal  information  that  indicates 
units  are  experiencing  problems  with  this  enemy  tactic.  Focusing  on  the 
collection  plan  will  confirm  or  deny  this  issue.  The  use  of  a  hypothesis 
statement  is  sometimes  more  beneficial  in  focusing  your  collection  efforts 
when  you  are  trying  to  confirm  that  a  corrective  has  been  applied  and  a 
change  of  behavior  is  anticipated  for  the  better.  For  example: 

Flypothesis:  The  implementation  of  counter  lED-defeat  mechanisms 
has  greatly  reduced  the  threat  of  IEDs  along  routes  traveled  by 
brigade  combat  teams. 

Figure  2-2  gives  a  sample  collection  plan  format  for  briefing  purposes  on 
one  issue.  Again,  the  plan  can  be  as  detailed  or  as  simple  as  the  observer 
desires.  The  sample  shows  the  issue  and  hypothesis,  lists  lead  or  key 
questions  to  ask,  and  describes  the  interviewees  or  individuals  who  will 
be  asked  the  questions.  Appendix  A  gives  an  example  of  a  very  detailed 
collection  plan,  and  Appendix  B  gives  an  example  of  interview  techniques. 

Direct  and  indirect  collections 

Now  that  you  have  a  collection  plan,  you  must  begin  to  decide  (if  you 
have  not  already  done  so)  how  you  intend  to  conduct  the  collection. 

An  LL  program  will  normally  use  two  different  types  of  collections, 
both  accomplishing  the  same  tasks  but  each  using  different  resources. 

The  direct  (sometimes  called  formal)  collection  effort  always  involves 
detailed  planning  and  is  generally  a  top-down  driven  process.  The  indirect 
(sometimes  called  informal)  collection  effort  uses  much  less  planning  and 
generally  fewer  resources,  but  is  typically  a  bottom-up  process. 

An  example  of  a  direct  collection  would  be  a  team  of  SMEs  assembled  and 
trained  on  the  collection  process  and  how  to  build  a  collection  plan,  focused 
on  specific  issues  generated  by  a  higher  authority,  and  deploying  overseas 
to  complete  a  30-day  collection  on  a  specific  organization.  An  example 
of  an  indirect  collection  could  be  as  simple  as  a  Soldier  in  a  military  unit 
submitting  an  observation  he  wrote  to  his  headquarters,  which  forwards  it 
up  the  chain  of  command,  eventually  getting  to  the  LL  organization.  This  is 
sometimes  called  an  unsolicited  observation.  It  may  or  may  not  be  tied  to 
an  existing  issue,  but  it  is  information  that  someone  deems  critical  enough 
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to  take  the  time  and  effort  to  submit  in  writing  from  a  lower  echelon. 
Although  it  is  always  challenging  to  get  organizations  to  submit  unsolicited 
observations,  they  are  typically  some  of  the  best  lessons  an  LL  organization 
receives. 


•  Issue:  Field  artillery  (FA)  core  competencies 

•  Hypothesis:  FA  personnel  participating  in  Operation  Enduring 
Freedom  (OEF)  and  conducting  nonstandard  missions  are  degraded 
in  their  F A/fire  support  core  competencies. 

•  Lead  Questions: 

o  What  nonstandard  missions  did  you  conduct? 

o  How  much  of  your  time  was  spent  on  other  than  FA  mission 
sets? 

o  How  much  time  are  you  spending  on  conducting  FA  missions? 

o  How  has  this  experience  impacted  your  primary  skills  of 
providing  fire  support? 

o  What  can  Fort  Sill  do  to  help  units  doing  nonstandard  missions? 

o  What  sort  of  predeployment  training  would  have  better 
prepared  you  for  nonstandard  missions? 

•  Interviewees:  FA  enlisted  Soldiers,  sergeants,  and  lieutenants  in  a 
maneuver  battalion;  FA  enlisted  Soldiers,  sergeants,  lieutenants  in 
an  FA  battalion  on  a  brigade  combat  team;  FA  battery  commanders; 
FA  battalion  S-3  officers;  and  FA  battalion  commanders. 

Figure  2-2.  Example  collection  plan  briefing 

Another  informal  collection  technique  may  be  simple  research.  If  a  qualified 
analyst  or  researcher  conducts  extensive  research  on  an  issue  and  finds  the 
right  information  and  lessons,  it  may  prevent  the  requirement  for  a  direct 
collection  effort,  saving  time  and  resources.  In  either  case,  the  researcher  or 
observer  in  each  instance  should  have  some  degree  of  expertise  in  the  area 
he  or  she  is  exploring,  hopefully  an  SME  who  can  provide  valuable  insights 
into  the  operation. 

The  only  reason  for  understanding  the  distinction  between  direct  and 
indirect  collections  is  to  realize  there  are  many  ways  to  get  LL.  Some  ways 
are  more  challenging  than  others.  It  may  sound  difficult  to  put  together 
a  direct  collection  team,  but  it  is  sometimes  harder  to  get  feedback  from 
the  field.  Anyone  involved  with  LL  programs  for  any  amount  of  time 
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understands  this.  If  timely  feedback  was  so  prevalent,  there  would  be  little 
need  to  deploy  direct  teams. 

Another  technique  for  collection  could  at  times  be  direct  or  indirect.  It 
concerns  the  use  of  embedded  teams  or  liaison  officers.  The  U.S.  Army 
has  effectively  used  this  concept  during  the  Iraq  and  Afghanistan  wars. 

They  brought  Reserve  officers  on  active  duty,  formally  trained  them  on 
collection  techniques,  helped  them  develop  a  collection  plan,  and  then 
deployed  them  into  theater  for  periods  of  six  months  to  one  year.  Each 
officer  was  embedded  with  a  unit  or  a  headquarters.  They  provided  vital  and 
timely  information  on  unit  operations;  tactics,  techniques,  and  procedures; 
and  lessons  learned.  This  effective  “push”  system  was  probably  a  direct 
collection  due  to  the  planning  and  resources  involved.  Other  techniques 
might  involve  the  use  of  survey  instruments,  or  the  positioning  of  liaison 
officers  with  various  organizations  that  have  their  own  LL  programs,  to 
maximizing  the  sharing  of  lessons  across  agency  boundaries. 

The  military  uses  a  very  instructive  process  as  outlined  in  Appendix  C 
called  the  after  action  review/report  (AAR).  You  would  think  that  this  is 
an  indirect  process  but  one  that  requires  a  great  deal  of  effort  and  time  to 
execute.  This  process  is  one  that  produces  some  of  the  best  and  most  timely 
lessons. 

•  The  after  action  review  is  a  verbal  discussion  held  at  the  completion 
of  an  operation  or  event  with  key  participants  to  determine  what 
happened,  what  worked,  what  did  not  work,  and  how  to  improve  for 
the  next  event. 

•  The  after  action  report  is  a  written  document  that  highlights  unit 
accomplishments  and  LL.  It  works  well  as  a  historical  reference  tool 
and  is  given  to  other  organizations  getting  ready  to  participate  in  the 
same  event. 

Although  the  U.S.  Army  developed  these  concepts,  with  some  modification 
they  could  be  adapted  for  interagency,  intergovernmental,  or  civilian  uses. 

Since  the  direct  collection  is  probably  the  most  complex  to  execute,  it  is 
beneficial  to  understand  the  process  to  execute  this  task.  Figure  2-3  depicts 
an  example  of  what  a  direct  collection  team  may  look  like.  This  example 
shows  four  observers/SMEs,  but  the  team  could  be  as  many  as  12  members 
if  the  issues  support  that  size.  More  than  12  are  cumbersome,  especially  if 
you  are  traveling  overseas. 
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Figure  2-3.  Generic  direct  collection  team  composition 

The  team  chief  has  the  responsibility  to  lead  the  team,  validate  the 
collection  plan,  review  observations  drafted  by  observers,  meet  regularly 
with  the  team  to  review  progress,  assist  in  writing  the  observation  report, 
and  conduct  any  briefings  required  to  introduce  the  team  or  provide  an 
out-briefing  when  the  team  departs  the  organization  it  is  observing.  The 
team  chief  should  be  of  sufficient  position  or  rank  to  “open  doors”  if  the 
requirement  exists.  In  the  military,  the  team  chief  is  usually  a  colonel. 

The  operations  officer  is  the  team  chief’s  deputy  and  his  “right-hand 
man.”  He  is  usually  a  full-time  member  of  the  LL  organization  with  prior 
experience  supporting  collection  teams.  He  performs  administrative  tasks 
such  as  equipping  the  team,  making  travel  arrangements,  and  scheduling 
interviews,  and  helps  the  team  chief  review  observations  for  correctness. 

Observers/SMEs  make  up  the  remainder  of  the  collection  team.  Observers 
have  the  responsibility  to  be  objective,  do  unbiased  reporting,  have  no 
personal  agendas,  and  be  good  writers,  since  they  are  responsible  for 
drafting  all  of  their  observations  for  the  observation  report  in  the  formats 
described  below.  Besides  the  use  of  a  survey  tool,  the  primary  way  to  get 
information  as  an  observer  is  through  an  oral  interview.  The  observer  should 
be  an  SME  as  well  as  the  person  he  is  interviewing.  Information  on  how  to 
conduct  an  oral  interview  is  in  Appendix  B. 

Training  for  any  collection  team  is  important.  Normally,  three  or  four 
days  of  training  before  the  team  departs  is  sufficient.  Subjects  taught  in 
this  “predeployment”  workshop  are  observer  duties;  security  training; 
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collection  plan  development;  and  any  training  required  on  issued  equipment 
such  as  laptop  computers,  cameras,  satellite  phones,  and  voice  recorders. 
However,  the  main  purpose  of  the  predeployment  workshop  is  to  finalize 
the  collection  plan  to  the  team  chief’s  liking.  It  is  also  important  to  conduct 
a  “post-deployment”  workshop.  The  primary  purpose  of  this  workshop  is  to 
draft  the  observation  report  as  described  below.  A  secondary  purpose  is  to 
return  issued  equipment,  finalize  travel  arrangements,  complete  a  security 
debriefing  to  ensure  there  is  no  compromise  of  classified  information,  and 
give  a  commander’s  or  director’s  briefing. 

Observation  reports 

The  result  of  a  direct  collection  effort  should  be  a  well-written  observation 
report.  Although  modification  may  be  necessary  to  correspond  to  any 
situation  or  desired  fonnat,  the  following  sections  are  included: 

•  Introduction:  The  introduction  may  discuss  the  mission  of  the 
collection  team,  dates  of  the  collection,  and  who  was  involved,  and 
typically  thanks  the  organizations  observed  for  their  support.  It  gives 
any  other  special  instructions  and  may  highlight  the  classification  of 
the  report. 

•  Executive  Summary  (EXSUM):  The  team  chief  leading  the  collection, 
with  input  from  the  individual  observers,  usually  writes  the  EXSUM. 

It  summarizes  the  results  of  the  collection  by  issue.  A  good  technique 
in  the  EXSUM  is  to  list  the  overall  top  three  or  four  issues  that  require 
immediate  attention.  Additionally,  the  EXSUM  should  cover  what  is 
working  well.  If  someone  reads  only  the  EXSUM,  they  should  get  a 
good  sense  of  what  the  collection  was  about  and  the  results. 

•  Chapters:  There  should  be  one  chapter  for  each  collection  issue.  Each 
observer  responsible  for  an  issue  should  write  his  or  her  own  chapter. 
The  chapter  should  start  with  a  brief  paragraph  summarizing  the 
major  findings  in  this  area.  It  should  then  list  each  observation  and  the 
associated  lessons,  recommendations,  and  any  best  practices  in  the 
prescribed  format. 

•  Appendices:  Miscellaneous  information  such  as  maps,  charts, 
checklists,  photos,  operation  orders,  and  briefing  slides  can  be 
included  here. 

You  can  use  several  different  formats  to  write  an  observation.  One  of  the 
easiest  to  use  is  the  issue-discussion-recommendation  or  observation- 
discussion-recommendation  format.  Another  format  the  U.S.  Army  uses  is 
the  following: 

•  Title:  The  topic  of  the  observation. 
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•  Description:  One  sentence  explaining  the  observation. 

•  Discussion:  The  major  analysis  of  the  problem  supported  by  facts  and 
examples. 

•  Lessons:  A  list  of  any  lessons  observed  to  share  with  other  units 
supported  by  the  discussion. 

•  Doctrine,  organization,  training,  materiel,  leadership  and 
education,  personnel,  facilities  (DOTMLPF)  Recommendations: 
Recommendations  for  corrective  actions  under  each  category  that 
applies  supported  by  the  discussion. 

The  joint  community  uses  the  following  observation  format: 

•  Topic/Issue:  The  title  of  the  observation. 

•  Observation:  One  sentence  explaining  the  observation. 

•  Discussion:  The  major  analysis  of  the  problem  supported  by  facts  and 
examples. 

•  Recommendations:  A  list  of  recommended  corrective  actions 
supported  by  the  discussion. 

•  Implications/Lessons:  A  statement  of  what  happens  if  the 
recommendations  are  not  adopted. 

NATO  uses  the  following  observation  format: 

•  Title:  Brief  but  specific  description  of  the  topic. 

•  Observation:  Brief  factual  description  of  what  happened. 

•  Discussion:  What  is  relevant  to  determining  the  root  cause?  (May 
include  history  of  the  event  and  the  what,  where,  when,  why,  who,  and 
how.) 

•  Conclusion:  Summary  of  the  underlying  issue  arising  logically  from 
the  observations  and  discussion. 

•  Recommendation:  Specific  actions  that  should  be  taken  and  by  whom 
to  deal  with  the  underlying  issue  presented  in  the  conclusion. 

NATO  also  has  a  format  for  a  best  practice: 

•  Title:  Topic  of  the  best  practice. 

•  Context:  Sets  the  context  for  the  best  practice  and  the  problem  area  or 
situation  that  it  resolves  or  improves. 
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•  Process:  Covers  the  main  body  of  the  best  practice;  an  explanation  of 
how  to  execute  the  best  practice,  and  what  processes  and  products  it 
requires. 

•  Benefit:  Justifies  why  it  should  be  classified  as  a  best  practice. 

(Provide  a  clear  reason  why  it  saves  lives,  time,  money,  resources, 
etc.) 

The  key  to  any  format  is  that  you  thoroughly  discuss  the  observation  using 
facts  and  examples  while  trying  to  avoid  opinions,  highlighting  any  lessons 
and  making  recommendations  to  fix  the  problem  if  you  have  the  expertise 
to  do  so.  When  dealing  with  LL,  do  not  mistakenly  write  a  truism  instead. 

A  truism  is  something  we  know  to  be  true  at  all  times.  At  one  point,  it  may 
have  been  an  LL  (a  long  time  ago),  but  over  time,  it  becomes  enduring.  For 
example,  IEDs  are  a  major  killer  on  the  battlefield.  Years  ago,  at  the  start  of 
the  war  on  terrorism,  this  was  a  lesson  learned;  today,  it  is  a  truism.  The  LL, 
in  this  example,  would  be  ways  to  defeat  IEDs. 

A  good  length  for  a  written  observation  is  one  to  two  pages.  Another 
thing  to  keep  in  mind  is  that  the  “shelf  life”  for  an  observation  is  about  six 
months.  This  means  that  anyone  who  attempts  to  use  the  observation  to 
justify  an  effort  after  about  six  months  needs  to  confirm  with  the  source  that 
the  observation  is  still  valid.  For  example,  maybe  the  source  organization 
determined  the  observation  will  not  be  corrected  for  whatever  reason,  or 
maybe  the  observation  was  corrected  and  it  is  no  longer  an  issue.  Using  an 
old  observation  or  observation  report  that  is  no  longer  valid  to  substantiate 
or  justify  a  project  without  making  sure  it  is  still  an  accurate  assessment  of 
the  issue  will  show  that  you  have  not  done  thorough  research. 

For  this  reason,  some  LL  organizations  prefer  to  keep  raw  observations 
as  internal  reports,  disseminated  only  to  those  who  have  a  need  for  the 
information.  One  technique  to  use  is  putting  a  statement  in  the  front  of  the 
report  that  “warns”  the  user  after  a  certain  period  of  time  to  make  sure  they 
verify  the  status  of  each  observation  before  citing  them  in  another  project. 

If  you  have  the  time  and  energy,  a  very  good  procedure  is  to  update  the 
status  of  each  observation  once  any  new  information  develops  concerning 
its  resolution.  This  requires  a  conscientious  staff  and  considerable  time  and 
effort  to  monitor  and  track  the  actions  associated  with  each  observation, 
most  of  which  may  not  directly  involve  the  LL  organization.  The  issues- 
resolution  process,  which  will  be  discussed  later,  can  provide  information  to 
support  this  effort. 

Make  every  effort  to  keep  the  observation  report  classified  at  the  lowest 
level  possible.  That  said,  it  is  the  responsibility  of  the  observer  to  ask  the 
interviewee  the  sensitivity  of  the  material  he  is  receiving.  If  there  is  any 
doubt,  always  have  the  unit  intelligence  officer  or  facility  security  specialist 
confirm  the  classification  of  the  information  in  question.  Be  especially 
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careful  with  photographs  taken  or  briefing  slides  with  embedded  pictures. 
Organizations  that  deal  with  classified  information  on  a  daily  basis  are 
sometimes  sloppy  in  their  handling  and  marking  of  documents. 

Finally,  out  of  courtesy,  it  is  appropriate  to  let  the  organizations  interviewed 
review  a  final  draft  of  the  observation  report.  In  some  instances,  they  may 
make  recommendations  that  clarify  what  was  given  in  the  interviews. 
Typically,  they  should  have  at  least  two  to  three  weeks  to  review  the 
observation,  a  time  you  need  to  factor  into  your  planning  sequence. 

What  you  must  avoid  is  the  tendency  for  the  reviewing  organization  to 
“correct  the  report.”  Any  LL  program  has  the  responsibility  to  present  only 
the  facts.  One  way  to  compromise  on  points  of  tension  is  to  include  the 
unit’s  opinion  as  an  appendix.  This  should  keep  both  sides  happy. 

Figure  2-4  highlights  the  LL  process  and  where  the  collect  function  fits. 


Issue 

Collect 

Observations 
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Figure  2-4.  The  LL  process  with  the  collect  function 


Analyze 

Analysis  is  a  process  used  to  thoroughly  understand  areas  of  activity 
identified  to  have  potential  for  improvement.  You  can  accomplish  this  action 
at  the  end  of  the  event  or  after  the  active  collection  has  been  completed; 
however,  a  certain  amount  of  analysis  may  be  done  throughout  the  LL 
process.  Transforming  the  raw  data  into  actionable  recommendations 
requires  a  systematic  process  to  examine  the  information  that  has  already 
been  collected  and  understand  why  or  what  contributed  to  the  need  for 
improvement.  The  level  of  analysis  may  be  determined  by  the  expertise 
or  resources  available  or  by  time  limitations  in  developing  a  final  product. 

In  some  cases,  it  may  be  more  important  to  conduct  a  surface  analysis  and 
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expedite  the  results  back  to  the  user  so  they  can  start  to  take  corrective 
action,  where  as  with  other  observations,  the  complexity  of  the  issue  or 
resolution  may  necessitate  a  more  detailed  analysis  and  explanation.  In 
either  case,  raw  observations  may  change  in  context,  content,  conclusion, 
and  applicability  during  analysis. 

Validation  of  the  observation 

During  this  part  of  the  analysis,  you  start  to  organize  the  data  you  have 
collected,  ensure  you  have  explored  every  possible  resource,  and  agree  on 
the  direction  and  method  the  analysis  should  follow. 

Frequency  of  occurrences.  Review  previous  observation  reports,  AARs, 

LL  databases,  and  other  reports  to  determine  the  frequency  and  conditions 
in  which  the  observations  have  occurred.  Is  there  a  solution  on  record? 

An  effort  should  be  made  to  determine  if  the  issue  has  been  previously 
identified  and  a  solution  initiated  but  not  yet  implemented. 

Understand  the  objective.  It  is  important  to  understand  where  to  focus 
your  analysis.  While  conducting  the  analysis,  it  is  easy  to  identify  additional 
issues  that  may  cause  you  to  stray  away  from  the  intended  objective  or  the 
customer’s  needs.  Consider  capturing  these  issues  and  setting  them  aside  for 
later  collection  efforts. 

Review  findings  with  the  host  organization  or  other  stakeholders.  If  not 

already  accomplished  during  the  collection  phase,  review  the  observations 
with  the  unit  or  organization  where  it  was  collected  to  determine  if  there 
were  any  unusual  or  contributing  conditions  for  the  issue.  Do  not  provide 
recommendations  at  this  point,  since  they  may  be  based  on  incomplete 
analysis. 

Analysis  of  observed  data 

This  is  the  step  where  the  analysts  start  to  analyze,  brainstorm,  and  dissect 
the  information  collected.  Use  different  perspectives  when  looking  at  the 
data  to  fully  understand  the  issues,  examine  each  piece  to  see  if  other  issues 
exist,  and  start  to  develop  the  full  story.  It  is  critical  to  discover  not  only 
what  happened  but  why  it  happened. 

•  Conduct  additional  research  on  the  observation(s).  This  may  require 
making  follow-up  interviews  or  phone  calls,  reviewing  AARs  or  other 
collection  reports,  and  searching  existing  LL  databases  for  related 
information.  Talking  to  other  SMEs  who  are  knowledgeable  of  the 
issue  may  also  provide  some  missing  information  or  confirm  the  data 
you  have  already  collected. 

•  Seek  expert  consultation  on  the  issue  to  help  you  understand  the 
data  you  have  collected,  identify  contributing  factors,  and  help  make 
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informed  recommendations.  In  most  cases,  you  will  not  be  an  SME  on 
the  particular  issue,  so  it  is  important  to  find  someone  who  can  answer 
questions,  review  the  observation  and  recommendations,  or  participate 
in  the  analysis  of  the  collected  data.  If  the  issue  is  a  piece  of  equipment 
or  system,  it  may  be  necessary  to  contact  the  manufacturer  or  designer. 
If  it  is  a  training  issue,  you  may  want  to  contact  the  training  center 
or  proponent  for  the  issue  to  see  what  is  being  taught  and  what  is 
missing. 

•  Organize  group  discussions  or  “murder  boards”  to  examine  the 
observations  in  more  detail,  identify  capability  gaps,  understand  the 
root  cause  of  the  finding,  and  determine  if  there  are  related  issues. 
Organize  these  discussions  with  someone  designated  to  keep  the 
discussion  on  track  and  the  analysis  of  data  moving  in  the  right 
direction.  Designate  someone  to  record  the  results  of  the  discussion. 
Displaying  the  data  on  a  screen  can  be  useful  for  the  group  to  view 
and  agree  on  the  outcome,  but  it  can  also  cause  the  discussion  to  get 
bogged  down  in  “wordsmithing”  and  delays  in  the  final  product. 

•  There  may  be  a  requirement  to  conduct  some  basic  statistical  analysis 
of  answers  you  received  in  questionnaires  or  interviews.  It  is  important 
to  interpret  and  sum  up  the  residts  for  the  purpose  of  the  analysis  and 
final  report.  Model  the  patterns  in  the  data  in  a  way  that  accounts  for 
randomness  and  uncertainty  in  the  observation.  It  may  be  useful  to 
display  this  infonnation  in  the  form  of  charts  or  graphs  in  the  final 
product  to  emphasize  the  frequency  of  an  observation,  gaps  in  a 
capability,  or  need  for  a  particular  solution. 

Corrective  actions/recommendations 

During  this  phase  you  start  to  organize  the  results  of  the  analysis  and 
determine  appropriate  recommendations.  If  you  do  the  collection 
and  analysis  correctly,  the  recommendations  should  be  intuitive.  The 
recommendations  should  say  what  needs  to  be  done  and  not  just  what  effect 
needs  to  be  achieved. 

•  Prepare  the  results  of  the  analysis  in  a  format  that  is  accurate  and 
easy  to  understand.  It  may  be  best  to  first  capture  the  data  on  briefing 
cards  or  in  briefing  slides  to  help  organize  the  main  points  in  a  logical 
order.  Add  to  the  observation  report  once  the  data  is  developed  and 
organized,  as  described  in  the  Collect  section  above. 

•  Provide  realistic,  actionable  recommendations.  Making 
recommendations  that  are  not  feasible  may  only  slow  down  the 
process  of  making  any  corrections.  Try  to  designate  the  agency  or 
organization  that  should  lead  the  corrective  action.  It  may  also  be 
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helpful  to  include  an  implication  statement  or  what  the  impact  would 
be  if  the  situation  is  not  corrected. 

•  Support  any  conclusions  with  interview  transcripts,  statistical  data,  or 
other  documents  that  will  provide  credibility  to  the  observation  and 
help  facilitate  the  solution  process. 

•  The  analytical  process  transforms  initial  observations  into  best 
practices  and  lessons  by  grouping  common  observations  to 
organizational  functions.  As  previously  mentioned,  the  military  uses 
the  DOTMLPF  construct  to  group  solutions;  however,  other  categories 
may  be  more  suitable  for  other  organizations. 

The  end  state  of  the  analysis  process  should  be  a  well-defined  list  of 
recommendations  or  potential  corrective  actions  with  sufficient  detail  to 
be  shared,  achieved,  and  entered  into  the  issues-resolution  process,  if  done 
correctly,  the  analysis  will  help  determine  the  action  plan  and  resources 
required  to  implement  the  corrective  actions. 

Figure  2-5  highlights  the  LL  process  and  where  the  analyze  function  fits. 


Share 

LL  programs  must  have  the  ability  to  share  and  disseminate  information 
to  be  effective.  Besides  the  ability  to  share  information,  the  LL  program 
must  be  able  to  determine  what  information  is  important  or  urgent  and 
how  rapidly  it  must  be  passed  to  other  organizations  that  could  benefit 
from  the  knowledge.  The  LL  program  must  have  a  process  and  medium 
to  do  this.  You  can  share  through  several  means,  such  as  printed  “hard 
copies,”  electronic  forms  like  e-mail  and  messaging,  collaborative  forums. 
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and  websites.  The  process  should  support  the  capabilities  of  the  medium. 
Additionally,  you  should  have  the  ability  to  handle  both  classified  and 
unclassified  material. 

Resourcing  of  the  LL  program  will  usually  determine  what  dissemination 
means  are  used;  however,  it  helps  if  the  resourcing  conforms  to  the 
information  required  by  the  target  audience  to  successfully  execute  their 
missions.  For  example,  printing  professional,  hard  copy  documents  can  be 
expensive  and  may  exceed  the  budget  limits  of  the  LL  organization,  but 
some  deployed  units  who  are  the  target  audience  may  not  have  access  to 
computers,  printers,  or  copiers  in  a  field  environment  and  need  hard  copy 
materials  to  do  their  mission  effectively.  In  the  future,  social  networking 
sites  similar  to  Facebook,  Twitter,  and  Linkedln  may  provide  additional 
capability  to  share  lessons  if  they  can  be  adapted  to  the  requirements  of 
an  existing  LL  program.  Finally,  it  is  imperative  that  all  LL  organizations 
within  a  common  community  be  linked  to  facilitate  the  flow  and  rapid 
exchange  of  information. 

Prioritization  of  Information 

The  key  to  information  or  LL  dissemination  is  a  rapid  sharing  process. 

This  requires  an  ability  to  rapidly  analyze  information  from  collections, 
determine  relevancy  and  timeliness,  and  gain  permission  from  leadership 
to  share.  The  challenge  in  this  process  is  the  quicker  you  need  to  get 
the  information  out,  the  more  risk  you  assume  in  conducting  a  thorough 
analysis  to  make  sure  you  are  drawing  the  correct  lessons. 

An  example  of  one  way  to  construct  a  rapid  sharing  process  is  depicted 
in  Figure  2-6.  In  this  example,  the  LL  program  has  created  timelines 
for  sharing  that  are  tied  to  the  urgency  of  the  information  and  a  medium 
to  disseminate  that  information.  The  terms  immediate,  urgent,  and 
routine  would  need  to  be  specifically  defined  to  meet  the  goals  of  the 
supported  organizations  and  their  mission.  For  example,  in  the  military,  an 
“immediate”  requirement  to  share  a  particular  lesson,  if  not  shared,  may 
result  in  the  injury  or  death  of  a  Soldier.  Flowever,  once  rapidly  shared, 
information  should  continue  through  the  analysis  process  and  eventually 
be  formally  vetted,  archived,  and  become  a  part  of  the  issues-resolution 
process,  if  it  rises  to  that  level  of  importance. 

You  can  publish  lessons  of  less  priority  as  articles,  reports,  bulletins,  and  so 
forth.  The  U.S.  Army  has  a  medium  where  any  Soldier  in  an  operation  can 
write  an  article  that  can  be  posted  on  the  LL  program  website  for  anyone 
to  read.  It  is  called  “News  From  the  Front.”  This  is  an  excellent  way  to 
post  timely,  thought-provoking  pieces  that  discuss  best  practices;  lessons 
learned;  and  tactics,  techniques,  and  procedures.  An  observation  report, 
discussed  in  the  Collect  section,  requires  a  longer  lead  time  to  produce 
because  it  is  usually  part  of  a  direct  collection  effort  involving  a  deliberate 
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planning  process.  Finally,  the  production  of  a  newsletter,  handbook,  or  other 
periodical  that  may  or  may  not  go  to  a  print  plant  for  production  usually 
requires  the  greatest  resources  and  the  longest  lead  times  to  produce.  They 
may  require  anywhere  from  three  to  six  months  to  develop  and  print; 
however,  they  additionally  serve  as  excellent  historical  documents  that  may 
be  very  beneficial  in  supporting  future  operations. 


t 
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Every  program  needs  a  process  to  rapidly 
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Figure  2-6.  Example  rapid  sharing  process 
Publications  and  automation 

As  previously  mentioned,  the  production  of  publications  and  the  ability  to 
disseminate  information  may  be  a  consideration  for  any  LL  program.  If  you 
cannot  share  the  information  you  collect,  it  is  virtually  useless.  Some  of  the 
major  functions  of  a  publications  capability  may  be  the  following: 

•  Manage  the  publications  process  from  first  draft  to  final  product  to 
ensure  consistency,  clarity,  and  correctness. 

•  Ensure  funding  is  available. 

•  Retain  necessary  editorial  and  graphic  artist  support. 

•  Accomplish  all  staffing  actions  before  final  publication. 
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•  Coordinate  with  the  printing  office  and  print  subcontractors  to  ensure  a 
quality  product  is  disseminated  in  a  timely  fashion. 

•  Work  with  proponent  and  supporting  agencies  to  coordinate  solutions 
and  exchange  ideas  on  production  efforts. 

Some  examples  of  categories  of  potential  publications  based  on  subject 
matter,  size,  and  frequency  are: 

•  Handbook:  A  “how-to”  manual  on  a  specific  subject  that  generally 
takes  a  longer  time  to  produce  than  most  other  publications. 

•  Newsletter:  Normally  less  intensive  than  a  handbook  to  prepare  but 
focused  on  a  specific  topic  of  interest  that  may  include  opinions  from  a 
wide  range  of  SMEs  used  to  stimulate  discussion  and  thought. 

•  Periodic  Bulletin:  Published  weekly,  monthly,  quarterly,  etc.  Less 
intensive  to  produce  than  a  newsletter  and  used  to  update  the  audience 
on  information  that  is  continually  evolving  or  changing. 

•  Article:  Prepared  by  an  author  to  highlight  a  specific  topic  of  interest. 

•  Special  Study:  A  publication  related  to  a  specific  operation,  exercise, 
or  subject,  such  as  a  country’s  history,  environmental  cautions,  cultural 
do’s  and  don’ts,  and  emerging  doctrine  or  policy. 

•  Observation  Report:  Published  after  the  completion  of  a  collection 
effort  that  summaries  the  observations,  lessons,  recommendations, 
and  best  practices  from  the  collection.  An  AAR  would  also  be  another 
form  of  an  observation  report  the  LL  organization  could  publish. 

Before  publishing,  one  challenge  to  sharing  information  is  the  degree  to 
which  the  LL  program  wants  to  edit  material.  There  will  be  times  when 
the  editorial  process  should  be  eliminated  or  modified  to  facilitate  the 
expeditious  release  of  information  critical  to  the  success  of  the  organizations 
being  supported.  As  the  saying  goes,  “perfection  is  the  enemy  of  good 
enough.”  The  editorial  process  should  not  become  so  burdensome  that  it 
prevents  a  document  from  being  published  in  a  timely  manner.  On  the  other 
hand,  if  time  is  not  an  issue,  a  professional  product  is  always  a  good  option. 

Automation  obviously  enhances  the  capability  to  share  information. 
Although  some  organizations  may  prefer  to  retain  a  paper-based  capability, 
that  capability  is  usually  expensive.  Others  may  prefer  a  purely  automated 
dissemination  system  using  electronic  distribution  techniques  such  as 
e-mails,  websites,  and  collaborative  sites.  Some  organizations  may  prefer 
both  capabilities,  paper  and  electronic.  However,  making  a  portable 
document  format  (PDF)  version  of  a  document  allows  for  easy  document 
exchange  and  is  low  cost  once  the  initial  investment  in  automation  is  made. 


26 


ESTABLISHING  A  LESSONS  LEARNED  PROGRAM 


You  must  give  consideration  to  what  extent  the  LL  program  wants  to  handle 
classified  information.  However,  if  your  goal  is  widest  dissemination,  it 
is  best  to  avoid  over  classification.  Some  LL  programs  have  the  ability  to 
create  databases  for  cataloging  observations  and  lessons  that  can  be  shared 
with  other  organizations.  Not  only  can  these  databases  store  observations, 
but  also  they  can  archive  the  source  material  and  reports  that  support  each 
observation.  For  example,  the  joint  military  community  uses  a  system  called 
the  Joint  Lessons  Learned  Information  System  (JLLIS),  and  the  homeland 
security  community  uses  a  system  called  Lessons  Learned  Information 
Sharing.  In  any  event,  the  challenge  of  publishing  information  and  sharing 
it  rapidly  reduces  by  the  sophistication  and  level  of  automation  existing  in 
most  organizations  and  agencies  today. 

Figure  2-7  highlights  the  LL  process  and  where  the  share  function  fits. 


Why  don’t  people  share? 

•  It’s  not  convenient. 

•  They  do  not  know  what  they  know. 

•  They  do  not  know  the  value  of  what  they  know. 

•  They  believe  knowledge  hoarding  is  job  security. 

•  They  do  not  get  credit  for  it. 

•  They  do  not  have  the  time. 

•  They  do  not  know  how. 
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•  They  do  not  know  who  is  interested. 

•  Sharing  of  issues  sometimes  leads  to  additional  work  to  fix. 

If  your  LL  program  can  begin  to  break  down  the  barriers  to  sharing  that  are 
listed  above,  then  you  will  be  on  your  way  to  increasing  the  productivity 
and  performance  of  the  organization.  Today,  there  are  organizations  and 
companies  that  have  done  just  that. 

Archive 

The  archive  function  is  a  broad  term  that  encompasses  several  required 
capabilities  that  most  LL  programs  desire.  The  ability  to  archive  information 
and  manage  records,  both  print  and  electronic,  allows  access  to  them  at 
a  future  date.  To  accomplish  this  task,  digital  repositories  (which  may  be 
called  digital  libraries  or  archives)  must  be  developed  to  store  information, 
facilitate  the  historical  preservation  of  information,  and  allow  users  to 
conduct  research.  Make  it  easily  retrievable  and  available  to  any  requestor 
by  providing  a  logical  system  for  storing  information.  Powerful  search 
engines  are  required  that  permit  rapid,  user-friendly  searches.  Finally, 
repositories  must  have  the  capability  to  store  and  guard  both  classified  and 
unclassified  data. 

Databases  and  websites 

Databases  and  websites  are  necessary  to  store  and  access  information.  Some 
examples  of  what  an  LL  program  may  desire  are: 

•  Stand-alone  applications  to  build  collection  plans. 

•  Stand-alone  applications  to  save  and  catalog  individual  observations 
resulting  from  collections. 

•  Stand-alone  applications  to  build  AARs. 

•  Digital  repositories  to  archive  current  information  and/or  older 
information. 

•  Programs  that  interface  with  other  LL  organizations. 

•  Organizational  websites. 

•  Collaborative  websites  to  share  information. 

The  LL  program  can  take  advantage  of  this  technology  to  increase  its 
efficiency  and  ability  to  access  information  in  a  rapid  manner.  One  of  the 
greatest  challenges  an  LL  organization  will  eventually  face  is  the  sheer 
volume  of  information  that  will  accumulate  in  its  archives.  You  will  need  to 
be  constantly  planning  ahead  and  determining  the  best  new  software  to  use 
to  upgrade  your  capabilities  and  be  open  to  new  technologies. 
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Research 

The  primary  reason  to  archive  information  is  to  have  the  ability  to  conduct 
future  research.  Research  today  no  longer  takes  place  on  library  shelves;  it 
takes  place  electronically  using  the  Internet.  Searchable  electronic  archives 
will  generally  be  of  three  types: 

•  Classified/Sensitive  repositories. 

•  Unclassified  repositories  with  access  limited  to  specific  groups,  such 
as  members  of  select  organizations. 

•  Public  repositories  available  to  all  Internet  users. 

Searchable  databases  provide  the  researcher  easy  access  to  the  information 
required.  This  capability  necessitates  the  use  of  highly  refined  search 
engines.  The  use  of  standardized  filing  and  naming  conventions,  protocols, 
and  accurately  tagging  documents  with  the  right  metadata  facilitates  the 
search.  The  metadata  describes  what  is  in  the  document  that  leads  you  to  the 
material  you  are  trying  to  find.  To  ensure  documents  are  properly  processed, 
you  may  want  to  establish  a  quality  assurance  check.  Additionally,  the  LL 
organization  may  require  the  capability  to  store  printed  material,  therefore 
requiring  a  “vault”  or  physical  repository  for  storage.  The  vault  is  also  a 
good  place  to  store  back-up  compact  discs  of  all  digital  material  archived 
electronically. 

It  is  also  beneficial  to  determine  criteria  for  inclusion  of  information  in  the 
repository.  Some  parameters  are  as  follows: 

•  The  item  is  from  an  official  approved  source. 

•  The  item  will  add  value  for  future  research. 

•  The  document’s  handling  restrictions  are  within  the  organization’s 
permissions. 

•  The  security  classification  is  within  the  organization’s  authority  to 
store. 

•  The  document  is  not  copyrighted,  unless  you  have  on  file  a  written 
permission  attached  to  the  document. 

The  archives  of  the  LL  organization  may  be  for  internal  use  only.  However, 
if  outside  agencies  are  permitted  to  gain  access  to  the  archives,  they  usually 
have  read-only  and  download  permissions. 

Finally,  the  way  information  is  cataloged  or  organized  must  be  simple  and 
easy  to  understand.  One  trap  many  organizations  fall  into  is  when  these 
procedures  are  designed  by  information  technology  personnel  and  not  by 
the  users.  To  be  simple  and  easily  understood,  the  user  must  have  input.  The 
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ability  to  quickly  find  what  you  are  looking  for  is  paramount  to  any  research 
capability.  Simplicity  is  the  key;  the  fewer  keystrokes  it  takes  to  find  a 
document  the  better. 

Historical 

Archived  information  serves  a  historical  purpose,  though  the  archiving 
and  adding  to  the  repository  should  take  place  as  soon  as  possible  after  the 
creation  of  the  document  or  information.  Years  from  now,  future  generations 
may  desire  to  study  the  accomplishments  of  your  organization.  They 
may  also  find  those  historical  examples  applicable  to  current  operations. 
Archiving  material  where  it  can  be  easily  retrieved  serves  this  purpose. 

For  example,  when  the  U.S.  Army  first  realized  it  would  be  involved  in 
counterinsurgency  operations  in  Iraq,  it  was  able  to  draw  on  documents 
archived  within  its  LL  program  from  the  Vietnam  War.  Since  these 
documents  all  existed  before  the  advent  of  the  computer,  over  the  years 
great  efforts  were  made  to  scan  this  information  into  the  Army  archives. 
Having  done  this,  great  time  and  effort  was  saved  when  it  became  necessary 
to  review  the  lessons  of  that  war  to  see  if  they  had  applicability  for  the  new 
conflict,  which  they  did. 

A  more  recent  example  concerned  Hurricane  Katrina  and  its  impact  on 
the  Gulf  Coast.  The  Center  for  Army  Lessons  Learned  (CALL)  searched 
the  Army  LL  program  archives  before  the  hurricane  hit  shore,  looking  for 
information  relating  to  the  1992  Hurricane  Andrew  disaster  in  south  Florida. 
CALL  then  rapidly  shared  the  lessons  contained  in  those  reports  with  Army 
units  preparing  to  deploy  and  support  Hurricane  Katrina  recovery  efforts. 

The  lessons  were  instrumental  in  assisting  units  in  their  planning  efforts. 

As  a  result,  archiving  information  for  future  generations  is  a  much-desired 
capability  within  any  LL  program.  It  will  mean  your  hard  work  may  get 
used  more  than  once. 

Requests  for  information  (RFIs) 

Every  LL  program  should  develop  a  system  to  answer  questions  or  RFIs 
submitted  to  it  from  individuals  outside  the  organization.  Providing  timely 
responses  to  questions  is  another  way  of  sharing  information.  Some 
examples  of  this  process  were  explained  in  the  Share  section  of  this  chapter. 
The  RFI  system  should  be  able  to  operate  in  both  classified  and  unclassified 
communication  networks.  The  system,  to  operate  efficiently,  should  include 
an  e-mail  system,  a  workflow  process,  a  document  management  application, 
a  structured  query  language  database,  Microsoft  Web  services,  and  a  Web 
interface.  Once  information  is  located  to  provide  a  response  to  the  RFI,  it 
can  be  sent  by  using  links  to  the  documents  with  the  requested  information. 
While  anyone  outside  your  organization  should  be  able  to  submit  an  RFI, 
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responses  are  provided  based  on  the  individual’s  rights  as  determined  by 
their  security  classification  or  clearance.  The  key  to  any  good  RFI  system  is 
providing  a  timely  response  with  some  degree  of  analytical  underpinnings, 
so  the  requestor  gets  the  information  in  a  manner  requiring  minimal  analysis 
on  his  part,  thereby  making  the  information  instantly  usable. 

Security 

If  your  LL  program  handles  classified  information,  it  must  have  the 
capability  to  review,  store,  and  archive  classified  publications.  Reviewing 
documents  to  determine  the  correct  classification  instructions  or  to  ensure 
that  documents  are  properly  marked  is  a  time-consuming  procedure, 
typically  requiring  a  separate  security  office  to  perform  these  duties.  It  will 
normally  require  dedicated  personnel  specifically  trained  to  execute  these 
tasks.  Due  to  the  sheer  volume  of  publications  coming  from  various  sources, 
there  can  be  mistakes  made  by  the  originators  whereby  briefings,  reports, 
documents,  etc.  are  improperly  marked,  which  could  lead  to  a  security 
violation  or  “spillage”  of  classified  data  over  an  unclassified  network. 

The  LL  program  must  have  the  capability  to  perform  electronic  “key  word” 
searches  on  each  document  received  to  determine  if  classified  information  is 
embedded  accidently  in  a  supposedly  unclassified  document.  Additionally, 
you  should  require  the  security  team  to  brief  and  debrief  all  direct-collection 
efforts,  so  personnel  are  trained  on  the  procedures  to  handle,  store,  and 
transport  classified  material.  Scrub  for  classified  information,  and  destroy 
or  properly  secure  all  notes  and  working  papers  once  the  collection  is 
completed.  Establish  rules  for  the  use  of  removable  media  such  as  thumb 
drives  and  compact  discs  early  on  to  avoid  loss  or  spillage  of  information 
not  intended  for  public  release.  In  addition,  all  laptop  computers  used  by  a 
collection  team  should  be  purged  of  all  information  once  the  team  returns 
and  is  done  with  their  use.  Finally,  consider  other  aspects  of  security, 
which  include  operational  security  to  protect  the  organization’s  plans  and 
procedures  and  physical  security  to  protect  the  facility,  equipment,  and 
personnel. 

Figure  2-8  highlights  the  LL  process  and  where  the  archive  function  fits. 
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Figure  2-8.  The  LL  process  with  the  archive  function 


Resolve 

The  most  challenging  component  of  any  LL  program  is  establishing  a 
process  to  legitimately  resolve  issues  once  analysis  is  completed  and  as 
early  and  quickly  in  the  process  as  possible.  Issues  resolution  beyond  the 
organization  is  challenging  for  several  reasons: 

•  Issues  resolution  requires  a  “forcing  function.”  This  means  it  typically 
needs  executive-level  support  and  involvement.  In  the  military, 

this  translates  into  general  officer  or  flag  officer  participation  in  the 
process.  Ideally,  you  should  involve  leaders  at  all  levels  who  are 
willing  to  dedicate  time  and  effort  to  resolve  issues  at  whatever  level 
they  occur. 

•  Someone  must  determine  which  issues  to  tackle,  and  are  they  willing 
to  invest  time  and  resources  (funding  and  people)  to  bring  about  a 
change  in  behavior.  Do  not  resource  LL  programs  to  accomplish  this 
task. 

•  To  resolve  any  issue  typically  requires  the  involvement  of  more  than 
one  agency  or  department. 

•  Many  issues  require  long  lead  times  to  resolve,  especially  if  the  issue 
requires  a  materiel  solution. 

•  You  can  make  a  strong  case  that  the  LL  organization  is  not  the  best 
organization  to  lead  this  process,  since  it  has  a  vested  interest  in 
the  collection  process  and  any  follow-on  efforts  to  determine  if  the 
corrective  actions  are  working. 


32 


ESTABLISHING  A  LESSONS  LEARNED  PROGRAM 


Whether  or  not  the  LL  program  has  responsibility  for  the  issues-resolution 
process,  the  organization’s  ability  to  change  behavior  by  implementing  a 
lesson  is  ineffective  unless  you  observe  that  change  and  a  determination 
made  that  the  lesson  is  learned.  In  other  words,  the  corrective  actions  have 
enhanced  performance.  To  do  this  requires  a  deliberate  process  to  commit 
resources,  make  decisions,  implement  those  decisions,  and  observe  the 
results.  If  the  process  is  cumbersome  and  too  difficult,  expect  the  results 
to  be  less  than  optimal.  If  the  process  takes  on  too  many  issues,  expect 
the  system  to  become  overwhelmed  and  collapse  on  the  sheer  weight  of 
taskings  to  agencies  that  simply  cannot  handle  the  additional  workload. 

It  is  better  to  resolve  your  top  three  issues  than  to  attempt  to  solve  the  70 
or  80  observations  from  the  last  collection.  Understand  that  the  level  that 
developed  the  issue  generally  has  the  highest  interest  in  resolving  it. 

Not  all  issues  require  a  formal  process  to  resolve.  In  the  military  context, 
if  unit  commanders  have  the  capability  to  correct  an  issue  internally, 
they  should  do  that.  It  is  possible  that  during  a  collection  effort  the  unit 
or  organization  becomes  aware  of  an  internal  problem  unknown  to  the 
leadership.  The  first  objective  is  to  handle  the  corrective  action  at  the  lowest 
level  possible.  The  issues  that  rise  to  the  next  level  of  attention  are  those  the 
unit  or  organization  are  unable  to  correct  internally.  Most  LL  programs  will 
focus  on  these  issues.  They  are  issues  that  require  assistance  from  sources 
outside  the  unit’s  chain  of  command.  For  example,  a  major  modification 
to  a  vehicle  system  would  be  an  issue  that  requires  outside  assistance  and 
generally  exceeds  the  ability  of  a  typical  military  unit  to  correct. 

The  issues-resolution  process  requirements  are  summarized  in  Figure  2-9. 


•  Senior  leader  involvement. 

•  Determination  to  commit  resources. 

•  Assign  responsibility. 

•  Develop  an  action  plan. 

•  Implement  corrective  action. 

•  Verify  the  lesson  is  learned. 

Figure  2-9.  Issues-resolution  process  requirements 

Senior  leader  involvement 

Senior  leader  involvement  or  executive-level  participation  in  the  issues- 
resolution  process  is  the  key  to  success.  Without  senior-level  leadership 
involvement  with  the  authority  to  task  agencies  to  work  issues  and 
reallocate  resources,  the  process  fails.  For  example,  successful  issues- 
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resolution  steps  in  the  U.S.  Army  required  involvement  at  the  three-star 
general  officer  level.  This  commander  had  the  ability  to  task  his  subordinate 
two-star  commanders  with  responsibility  for  their  respective  service 
branches  and  combat  development  processes  to  drive  changes  to  doctrine, 
training,  education,  or  materiel.  If  you  cannot  fix  responsibility  at  the 
appropriate  level,  then  the  process  becomes  the  end  state.  The  process  is  not 
as  important  as  the  product.  The  product,  in  this  case,  is  a  corrective  action 
that  when  implemented,  changes  behavior.  To  develop  that  product  is  not 
an  easy  task  and  always  requires  someone  to  take  ownership  at  a  level  able 
to  direct  others  to  complete  actions  and  hold  them  accountable  for  their 
progress. 

Determination  to  commit  resources 

If  you  want  an  effective  LL  program  and  an  issues-resolution  process,  you 
should  at  some  point  present  the  issues  gathered  and  analyzed  to  a  leader 
who  can  assist  in  the  prioritization  process  and  make  decisions  to  commit 
resources  to  solve  problems.  There  will  be  few  if  any  corrective  actions 
that  will  not  require  the  expenditure  of  some  resource,  be  it  time,  additional 
manpower,  or  money.  You  will  quickly  determine  you  cannot  fix  everything. 
Obviously,  you  can  accomplish  some  things  with  fewer  resources  than 
others.  For  example,  a  change  to  written  doctrine  or  policy  typically 
requires  less  effort  and  expenditure  of  resources  than  a  materiel  solution  for 
a  piece  of  equipment  or  new  operating  system. 

To  determine  where  to  place  your  efforts  requires  a  prioritization  process 
or  an  ability  to  rank  the  issues  from  most  important  to  least  important.  This 
may  involve  a  risk  assessment.  Normally,  risk  assessments  are  subjective, 
and  qualitative  analysis  becomes  the  norm.  In  some  instances,  a  military 
commander,  based  on  his  experiences,  professional  judgment,  and  “gut” 
instincts,  can  make  a  good  assessment  of  risk.  In  other  instances,  the 
assessment  requires  the  collective  input  and  wisdom  from  a  group  of  SMEs 
or  other  professionals.  For  example,  the  military  sometimes  uses  a  “council 
of  colonels”  to  assist  in  making  recommendations  to  more  senior  officers  for 
final  decision.  Civilian  organizations  may  convene  a  director-level  working 
group  or  use  a  board  of  directors.  Eventually,  you  reach  a  consensus  or 
you  are  directed  where  to  place  your  efforts  based  on  a  combination  of  risk 
levels  and  resource  availability. 

The  key  is  to  focus  your  efforts  on  the  most  important  things  that  need  to  be 
fixed,  where  you  can  get  the  greatest  “bang  for  the  buck.”  For  example,  in  a 
military  organization,  every  effort  is  made  to  fix  those  issues  that  may  have 
life-or-death  consequences;  they  are  considered  high-level  risks.  Flowever, 
the  challenge  is  sometimes  greater  in  prioritizing  issues  that  do  not  have  that 
clear  distinction.  The  goal  of  this  requirement  is  to  determine  what  you  have 
the  ability  to  correct  and  then  begin  to  develop  a  plan  to  do  so. 
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Assign  responsibility 

The  person  in  charge  with  the  authority  to  commit  resources  needs  to  pick 
a  lead  agency  to  work  the  issue.  In  most  instances,  there  will  be  supporting 
agencies  that  must  assist  the  lead  agency.  It  is  rare  in  today’s  operational 
environment  that  only  one  agency  can  fix  a  problem  by  itself.  Additionally, 
the  LL  program  should  not  be  the  lead  agency.  The  LL  program  supports 
the  resolution  effort  by  providing  guidance  and  information  relating  to 
ongoing  or  subsequent  collection  efforts  that  may  inform  or  assist  the  lead 
agency  in  its  work.  Typically,  the  LL  program  does  not  have  the  specific 
subject  matter  expertise  or  the  resourcing  to  resolve  major  issues  of  a  very 
important  nature  that  require  a  formal  action  plan.  Its  focus  should  remain 
on  continued  collection  efforts,  analysis  of  information,  and  sharing  the 
lessons  with  other  organizations  that  may  experience  the  same  challenges 
as  the  agencies  working  to  correct  the  issue.  As  stated  previously,  the 
first  place  an  issues-resolution  process  will  fail  is  by  not  having  senior 
leadership  involved  in  the  process  from  the  start.  The  second  place  it  will 
fail  is  by  not  assigning  responsibility  to  a  single  agency  to  be  the  lead  for  a 
corrective  action. 

Develop  an  action  plan 

Once  a  determination  is  made  to  solve  an  issue,  an  action  plan  should  be 
developed  by  the  lead  agency  with  responsibility  to  work  the  issue.  You 
should  approve  the  plan,  usually  at  the  level  that  has  authority  to  assign 
resources.  The  action  plan  summarizes  the  issue,  weighs  the  risk,  outlines  a 
way  ahead  or  timeline  for  resolution  of  the  issue,  and  assigns  responsibility. 
The  plan  can  also  specifically  assign  the  resources  required.  It  can  be  as 
complex  or  as  simple  as  needed.  The  example  at  Figure  2-10  shows  a 
simple  “quad”  sheet  format  for  briefing  purposes.  Establish  more  elaborate 
and  detailed  timelines  in  separate  enclosures. 

Risk  levels  are  more  important  in  initial  decision  briefings  to  determine 
prioritization.  If  you  use  risk  levels,  you  must  define  what  each  risk  level 
means.  Once  those  are  established  and  accepted,  you  can  replace  them 
with  other  necessary  information  categories,  such  as  required  resources/ 
constraints,  essential  tasks,  or  proposed  end  state.  You  can  use  any 
combination  of  categories  to  convey  the  necessary  information  and  satisfy 
the  desires  of  the  briefing  audience,  as  required.  The  goal  of  the  action  plan 
is  to  track  the  progress  and  milestones  of  the  lead  agency  toward  resolution 
of  the  issue. 
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Issue  Title 

•  Issue  Statement: 

•  Risk  Level  if  Not  Resolved  and 

Why: 

^  High 

•  Lead: 

Medium 

•  Assists: 

Law 

•  Issue  Discussion: 

•  Way  Ahead/Timeline  to  Resolve: 

•  Required  Resources: 

Figure  2-10.  Example  quad  sheet  action  plan 

Implement  corrective  action 

Implementation  of  a  corrective  action  is  seldom  the  responsibility  of 
the  LL  program.  This,  again,  is  one  reason  why  senior  leadership  must 
be  involved  in  the  process.  In  some  instances,  training  may  be  required 
before  implementation.  Implementing  a  corrective  action  could  be  a  long 
and  deliberate  procedure  well  beyond  the  ability  of  any  LL  program  to 
manage  or  control,  or  it  could  simply  be  a  change  to  an  existing  policy  or 
procedure.  Like  any  solution  to  a  problem,  there  will  usually  be  costs  to 
implement  in  terms  of  time,  people,  and/or  money.  For  example,  the  U.S. 
Army  categorizes  change  using  seven  categories:  doctrine,  organization, 
training,  materiel,  leadership  and  education,  personnel,  and  facilities 
(DOTMLPF).  However,  changing  doctrine  is  generally  easier  to  do  than 
changing  an  organization  design,  etc.  The  LL  program  exists  to  facilitate 
the  change  process  by  providing  observations  and  lessons  for  analysis  and 
consideration  for  incorporation  into  an  issues-resolutions  process  whose 
goal  is  to  enhance  organizational  performance. 

Verify  the  lesson  is  learned 

Verifying  the  corrective  action  or  lesson  is  “learned”  once  again  comes 
under  the  oversight  of  the  LL  program.  It  will  require  an  additional 
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collection  effort  to  validate  the  results.  Whether  that  feedback  is  gained  by  a 
direct  or  indirect  collection  effort,  the  process  is  not  complete  until  someone 
determines  there  is  a  change  in  behavior  as  a  result  of  the  corrective  actions 
applied  to  the  problem.  The  LL  program  must  consider  these  additional 
collections  as  part  of  its  overall  responsibilities  and  plan/budget  for  them. 
This  requirement  to  verify  the  lesson  is  learned  is  one  reason  why  the  LL 
program  should  remain  as  a  “player”  and  not  the  “coach”  of  the  issues- 
resolution  process.  By  doing  this,  the  LL  program  can  remain  the  “honest 
broker”  within  the  process. 

You  can  make  a  case  that  in  the  military  community  a  lesson  is  never 
really  learned  because  of  many  factors  such  as  personnel  turnover,  level 
of  training,  and  changing  operational  environment.  From  the  perspective 
of  a  doctrinal  or  materiel  solution,  this  may  not  be  as  true.  However,  what 
this  does  point  out  is  the  human  dimension  of  military  operations.  When 
an  organization  is  centered  around  people,  as  is  the  U.S.  Army,  the  ability 
to  change  behavior  is  an  ongoing  process.  This  at  times  can  frustrate  an  LL 
program,  because  it  may  begin  to  see  the  same  mistake  made  over  and  over 
again,  even  after  a  corrective  action  was  applied.  This  should  not  discourage 
the  program  or  indicate  it  is  ineffective.  Any  good  LL  program  understands 
this  and  adjusts  accordingly  by  continuing  to  provide  the  most  accurate 
information  possible,  so  decision  makers  can  decide  how  they  want  to 
prioritize  and  focus  resources  to  resolve  new  deficiencies  while  maintaining 
an  ability  to  observe  the  success  of  implementing  past  corrective  actions. 
The  most  important  point  to  remember  is  that  an  observation  does  not 
become  an  LL  until  behavior  has  changed. 

Figure  2-11  below  highlights  the  entire  LL  process  by  function. 


Figure  2-11.  The  LL  process 
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Assess 

There  are  several  ways  to  determine  if  the  LL  program  is  effective.  You  can 
evaluate  any  LL  program  by  the  expenditure  of  resources  against  the  desired 
results.  It  is  difficult  to  determine  through  quantitative  analysis.  However, 
there  are  some  ways  to  evaluate  the  effectiveness  of  an  LL  program  that 
may  validate  and  justify  its  need.  Assessment  of  LL  effectiveness  can  be 
broken  into  several  components:  organization  behavior,  organization  or  unit 
performance,  and  mission  effectiveness. 

Observed  behavior 

If  the  training,  education,  or  testing  plan/program  is  not  exhibiting  the 
same  errors  or  mistakes  identified  and  addressed  by  the  LL  program,  it 
is  reasonable  to  conclude  that  the  LL  program  has  had  a  positive  effect. 
Some  factors  that  may  affect  this  are  time  required  for  the  lesson  to  be 
disseminated  and  time  for  changes  in  the  training  plan  to  take  effect.  You 
can  discover  observed  behavior  in  plan  and  program  reviews,  event  AARs, 
project  reports,  retrospectives,  and  summaries. 

Lessons  learned  (and  applied)  are  re-examined  during  active  collection  by 
the  LL  activity.  Collection  teams  can  look  for  older  issues  and  determine 
if  the  problem  still  exists  or  if  the  problem  was  solved  by  information 
disseminated  in  the  LL  program. 

Organization  or  unit  performance 

Verify  LL  and  lessons  applied  by  charting  organization  or  unit  performance 
against  established  measures  of  performance  (MOPs)  for  mission-essential 
tasks  that  must  be  performed  by  the  organization  or  unit.  MOPs  are  simply 
metrics  of  tasks  the  organization  or  unit  must  be  able  to  perform  with  its 
organic  resources  to  accomplish  its  mission.  Examples  of  this  are  enough 
trained  drivers;  personnel  properly  trained  and  certified  on  equipment;  and 
the  ability  to  perform  operational,  logistical,  and  administrative  tasks  in 
keeping  with  organization  policy,  regulations,  and,  if  applicable,  civil  laws. 

Mission  effectiveness 

Lesson  learned  and  lessons  applied  effectiveness  can  also  be  judged  by 
charting  organization  or  unit  performance  against  measures  of  effectiveness 
(MOEs).  MOEs  are  more  goal  and  objective  centric.  The  chief  criteria 
for  MOE  success  are  the  questions,  “Is  the  organization  effective  in  the 
accomplishment  of  its  mission,  and  is  it  supporting  the  overall  goals, 
objectives,  and  missions  of  the  parent  organization?”  If  an  organization 
has  increased  mission  effectiveness  over  previous  performances,  you  can 
attribute  it  to  lessons  learned  and  applied  from  the  LL  activity.  However, 
there  may  be  other  factors  that  account  for  an  increase  in  mission 
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effectiveness;  therefore,  additional  collection  may  be  required  to  verify  the 
effectiveness  of  the  LL  program. 

Establishing  written  procedures 

Once  you  have  decided  the  size  and  scope  of  your  LL  program,  it  is  a  good 
idea  to  formalize  your  program  by  developing  a  set  of  written  guidelines, 
responsibilities,  policies,  and  procedures.  The  military  would  call  this  a 
standing  operating  procedure.  Whatever  term  is  used,  it  is  necessary  to 
standardize  your  procedures  in  writing  for  several  reasons.  First,  it  is  one 
way  to  get  decision  makers  and  leadership  to  agree  to  the  specifics  of  the 
program.  Second,  it  is  a  document  that  everyone  can  read  and  become 
familiar  with  all  procedures.  Third,  it  allows  participants  to  understand  their 
specific  responsibilities.  Fourth,  it  can  be  used  to  support  funding  decisions, 
since  you  will  have  established  procedures  approved  by  your  executive 
leadership.  Finally,  it  gives  you  a  plan  that  can  be  adjusted  over  time  to 
accommodate  changing  priorities  or  direction  in  your  organization  or  unit. 
The  goal  of  written  procedures  is  to  enhance  unity  of  effort. 
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Chapter  3 

Organizational  Considerations 

Do  you  ever  have  problems  finding  the  answers  you  need? 

Have  you  ever  wanted  to  find  someone  who  has  done  this  before  and  could 
give  you  some  pointers? 

Do  you  ever  think  there  has  to  be  an  easier  way  to  do  something? 

How  much  time  have  you  spent  looking  for  a  document  on  the  Web  and  not 
been  able  to  find  it? 

General 

If  you  are  determined  to  create  a  lessons  learned  (LL)  capability  within 
your  organization,  you  have  one  basic  question  to  ask  yourself:  What  is 
the  purpose  of  my  LL  program?  Answering  this  question  should  help  you 
determine  what  functions  from  Chapter  2  you  require.  Most  organizations 
also  have  constraints,  be  they  people,  time,  funding,  etc.  These  limitations 
will  impact  the  extent  to  which  an  organization  can  devote  resources  to 
creating  what  is  probably  a  new  program.  For  example,  the  "low-budget” 
approach  might  be  the  implementation  of  a  program  that  focuses  only  on  the 
collect-analyze-share  functions.  The  “high-budget”  approach  may  focus  on 
the  collect-analyze-share-archive-resolve  functions. 

Organization  structure  also  plays  a  role.  Organizational  structure  is  a  key 
component  for  sustainability  of  information  empowerment.  You  should 
address  organizational  considerations  from  the  very  beginning,  allowing 
the  organizational  transformation  to  occur  gradually  and  positioning  the 
company  or  military  unit  to  fully  leverage  newly  acquired  information  once 
it  is  available. 

More  often  than  not  an  inherent  challenge  lies  within  corporate 
organizational  structures  to  accommodate  the  functional  responsibilities 
of  everyday  business.  Departments  may  need  to  work  collectively  only 
when  they  are  focused  on  specific  initiatives.  Unfortunately,  LL  systems 
are  usually  ineffective  because  they  invariably  introduce  new  processes, 
when  instead  they  should  be  embedded  into  the  processes  they  are  meant  to 
improve.  An  LL  program  will  thrive  under  the  following  conditions: 

•  Where  people  continually  expand  their  capacity  to  create  the  results 
they  truly  desire,  where  new  and  expansive  patterns  of  thinking  are 
nurtured,  where  collective  aspiration  is  set  free,  and  where  people  are 
continually  learning  to  see  the  whole  together. 
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•  In  situations  of  rapid  change,  only  those  that  are  flexible,  adaptive, 
and  productive  will  excel.  For  this  to  happen,  organizations  need  to 
discover  how  to  tap  people’s  commitment  and  capacity  to  learn  at  all 
levels. 

•  Where  systemic  thinking  is  the  cornerstone  of  the  learning 
organization,  as  the  training  that  integrates  all  others  to  comprehend 
and  address  the  LL  as  a  whole. 

Military  organizations  with  distinct  levels  of  command  and  well- 
defined  hierarchies  admittedly  have  fewer  challenges  when  it  comes  to 
implementing  and  integrating  LL  programs.  Militaries,  which  are  faced 
with  life-and-death  decisions,  unlike  corporations,  which  focus  primarily  on 
making  a  profit,  have  real  motivational  reasons  to  learn  from  their  mistakes 
to  prevent  the  needless  loss  of  life  and  equipment,  to  say  nothing  about 
winning  their  nations’  wars.  From  that  point  of  view,  corporations  might 
benefit  from  understanding  the  success  the  military  has  achieved  integrating 
LL  processes  into  daily  operations. 

Questions 

Following  are  a  few  questions  to  help  you  determine  the  purpose  and  scope 
of  your  LL  program: 

•  Is  your  organization  willing  to  expose  problems  or  mistakes  for  the 
betterment  of  the  collective  whole,  and  can  you  do  this  in  a  way  that 
does  not  intimidate  employees  or  workers? 

•  Do  you  want  a  centrally  managed,  formal  program,  or  do  you  want 
a  decentralized  program  that  places  more  responsibility  at  the  lower 
levels  for  LL  implementation? 

•  Who  is  the  best  person  or  office  to  put  in  charge  of  your  LL  program? 

•  How  do  you  intend  to  identify  issues  for  analysis? 

•  What  ways  does  your  organization  have  to  share  information? 

•  After  you  identify  issues  and  analyze  them,  do  you  want  to  have 
a  way  to  recommend  corrective  actions  and  then  monitor  their 
implementation? 

•  When  sharing  information,  what  are  your  release  procedures  for  this 
information  beyond  your  office  or  organization? 

•  What  additional  automation  requirements  are  necessary  to  support  the 
purpose  of  your  program? 

•  How  will  you  know  your  LL  program  is  working? 
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Management  and  Coordination 

The  common  practice  is  to  perform  LL  at  the  end  of  a  project  phase; 
however,  timing  is  everything.  Do  not  wait  for  the  phase  or  project  to  be 
over;  do  LL  early  and  often.  LL  should  not  be  an  afterthought  but  a  key 
component  of  all  project  management  processes.  (See  Appendix  C,  Military 
After  Action  Reviews/Reports.) 

Initiating  and  planning: 

•  Identify  similar  projects. 

•  Gather  useful  information. 

•  Determine  mitigation  strategies,  if  applicable. 

•  Incorporate  LL  into  the  new  project  plan. 

Identify  projects  from  which  to  gather  data: 

•  Look  for  common  threads  to  projects  similar  in  nature. 

•  Common  topics. 

•  Common  learning. 

Gather  useful  information: 

•  Interview  people. 

•  Review  historical  project  data. 

Tips  for  best  results: 

•  Capture  LL  as  close  as  possible  to  the  learning  opportunity  (e.g.,  after 
an  issue  has  been  resolved,  change  in  scope  has  occurred,  or  a  risk  has 
been  mitigated). 

•  Identify  project  management  processes  that  can  be  improved  because 
of  LL,  and  make  the  improvements. 

•  Maintain  an  LL  log  throughout  the  life  of  the  project. 

Learning  in  organizations  happens  in  two  ways: 

•  Learning  by  individuals. 

•  Investing  in  team  members  who  have  knowledge  the  organization  did 
not  have. 
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Key  take-aways  for  any  organization: 

•  You  must  gather  and  implement  valuable  knowledge  to  be  reused  for 
the  betterment  of  future  project  success. 

•  LL  are  gathered  and  implemented  during  and  not  just  at  the  end  of  the 
project. 

•  Timing  is  crucial  in  gathering  LL. 

•  You  must  build  LL  documents  that  can  be  effectively  used  for  future 
projects. 

•  Plan  for  LL. 

Knowledge  Management 
What  is  knowledge? 

Knowledge  is  more  than  data  or  information.  Knowledge  comprises  a 
range  of  strategies  and  practices  used  in  an  organization  to  identify,  create, 
represent,  distribute,  and  enable  the  adoption  of  insights  and  experiences. 
Such  insights  and  experiences  comprise  knowledge,  either  embodied  in 
individuals  or  embedded  in  organizational  processes  or  practice. 

Knowledge  management  (KM)  efforts  typically  focus  on  organizational 
objectives,  such  as  improved  performance,  competitive  advantage, 
innovation,  the  sharing  of  LL,  integration,  and  continuous  improvement  of 
the  organization. 

Types  of  knowledge 

There  are  three  types  of  knowledge: 

•  Tacit:  Personal  knowledge  that  resides  within  an  individual,  which 
relies  on  experiences,  ideas,  insights,  values,  and  judgments. 
Knowledge  that  is  resident  within  the  mind,  behavior,  and  perceptions 
of  individuals.  Knowledge  developed  and  internalized  by  an  individual 
over  a  long  period  of  time,  incorporating  so  much  accrued  and 
embedded  learning  that  its  rules  may  be  impossible  to  separate  from 
how  an  individual  acts. 

•  Explicit:  You  can  convey  formal  knowledge  from  one  person  to 
another  in  systemic  ways  such  as  documents,  e-mails,  and  multimedia. 
This  is  knowledge  easily  codified  and  conveyed  to  others. 

•  Organizational:  The  combination  of  critical  data,  information,  and 
knowledge  with  collective  intellect,  which  enables  an  organization  to 
learn  from  experiences,  innovate,  make  decisions,  create  solutions, 
perform  tasks,  or  change  positions. 
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Eighty  percent  of  an  organization’s  knowledge  is  tacit.  Organizations 
must  value  and  capture  both. 

KM  efforts  overlap  with  organizational  learning  and  may  be  distinguished 
from  that  by  a  greater  focus  on  the  management  of  knowledge  as  a  strategic 
asset  and  a  focus  on  encouraging  the  sharing  of  knowledge.  KM  efforts  can 
help  individuals  and  groups  share  valuable  organizational  insights,  reduce 
redundant  work,  avoid  reinventing  the  wheel  per  se,  reduce  training  time 
for  new  employees,  retain  intellectual  capital  in  an  organization  during 
employee  turnover,  and  adapt  to  changing  environments  and  markets.  The 
challenge  for  any  LL  program  is  to  find  a  way  to  get  people  to  share  tacit 
knowledge  among  themselves. 


Knowledge  is  information 
analyzed  to  provide 
meaning  and  value,  or 
evaluated  as  to  implications 
for  the  operation 


Data  is  the  lowest  level  of  information  on  Ihe  cognitive 
hierarchy  Data  is  rarely  useful  until  It  is  processed  to  give 
it  meaning 


Information  13  dele  mat 
have  been  processed  to 
provide  further  meaning. 


Data 


Judgment  \ s  a  purely 
human  skill  It  is  based  on 
experience,  expertise,  and 
intuition. 


Coqnft/on- trie  act  of 
learning,  ofintegrating 
from  various  pieces  of 
information — allows 
supervisors,  commanders 
and  staffs  to  generate 
knowledge. 


Processing  includes 


filtering,  fusing, 
formatting,  organizing, 
collating,  correlating, 
plotting,  translating, 
categorizing,  and 
arranging 


Figure  3-1.  Cognitive  hierarchy 


Knowledge  is  meaningfully  structured  and  based  on  experience.  Some 
is  usable  as  the  basis  for  achieving  understanding  and  making  decisions. 
Other  knowledge  forms  the  background  against  which  administrators  or 
commanders  make  decisions.  The  cognitive  hierarchy,  shown  in  Figure 
3-1,  portrays  the  place  of  data,  information,  and  knowledge  in  developing 
understanding.  This  figure  also  shows  the  roles  of  KM  and  information 
management  in  this  development. 
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Lessons  Learned  Examples 

In  2007,  Mattel  Chairman  and  CEO  Bob  Eckert’s  organization  learned  some 
valuable  lessons  after  issuing  a  major  toy  recall  just  prior  to  a  significant 
shipping  month.  Faced  with  angry  consumers  and  lawmakers,  Mattel  had 
to  take  responsibility  for  the  recall.  Regrettably,  the  media  also  chose  to 
saturate  the  news  with  the  Mattel  story  as  the  lead.  Mattel  had  to  act  fast  to 
reclaim  traction  with  consumers  and  stakeholders,  which  ended  in  praise 
for  the  way  the  company  handled  the  situation.  Bob  Eckert  states  the  most 
important  thing  in  a  crisis  situation  is  to  be  straight  about  it  and  be  quick. 
Below  are  five  key  LL  from  the  crisis  at  Mattel: 

•  Always  act  fast.  Confront  the  issue;  do  not  hide  from  it.  In  Mattel’s 
case,  the  company  was  very  public  about  the  recalls,  and  the  CEO 
even  issued  a  public  apology.  A  quick  reaction  makes  it  easier  for 
companies  to  cope  with  and  take  control  of  the  situation.  Reacting 
quickly  helps  companies  score  “bonus  points”  with  the  public,  slightly 
reducing  the  negative  impact  that  a  recall  has  on  the  company’s 
reputation.  When  companies  are  slow  to  react  or  spend  most  of  their 
time  placing  blame  on  others,  the  public  reacts  negatively,  criticizing 
companies  for  their  negligence  and  irresponsibility.  A  quick  reaction 
won’t  solve  all  of  the  problems,  but  failing  to  do  so  will  open  up  a  new 
can  of  worms  to  deal  with. 

•  Keep  an  eye  on  your  supply  chain.  To  save  on  costs,  Mattel  has 
shipped  manufacturing  overseas  to  China.  Having  multiple  offices 
and  operation  sites  makes  it  difficult  to  keep  an  eye  on  day-to-day 
operations.  According  to  the  Financial  Times  Press  article  “Trouble 
in  Toyland:  New  Challenges  for  Mattel-and  ‘Made  in  China’,”  one 
of  the  main  issues  in  the  lead  paint  crisis  at  Mattel  was  that  Chinese 
contractors  had  subcontracted  the  painting  of  the  toys  to  another 
company  that  used  inferior  and  unauthorized  products.  A  lot  of 
companies  get  caught  in  similar  traps. 

•  Take  responsibility.  Be  the  bigger  person  and  take  the  blame  — 
public  finger  pointing  is  not  going  to  get  you  anywhere.  In  the  Reuters 
article  “Mattel  Sued  Over  Toy  Recall,”  it  was  reported  that  Mattel’s 
CEO  stated  that  the  company  was  increasing  the  aggressiveness  of 
toy-testing  methods,  which  would  likely  result  in  additional  recalls  as 
a  precautionary  measure. 

•  Tighter  regulations  and  inspections.  In  the  Wall  Street  Journal 
article,  “Mattel  Settles  Suit  Over  Lead  in  China-Made  Toys,”  author 
John  Kell  writes:  “Toy  makers  were  hurt  by  a  number  of  product 
recalls  in  2007,  leading  to  millions  of  dollars  in  costs  for  testing,  legal 
expenses,  advertising,  and  product  returns.  Mattel  recalled  millions  of 
toys  that  year,  including  those  produced  under  licenses  for  characters 
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including  Elmo,  Big  Bird,  Barbie,  and  Polly  Pocket.  The  issue  later  led 
to  mandatory  federal  toy-safety  standards,  which  included  testing  and 
tough  new  regulations  for  lead  and  chemicals  in  products  intended  for 
children  under  12.” 

•  Take  action  and  communicate.  During  a  crisis,  such  as  the  one 
experienced  by  Mattel,  a  lot  of  business  leaders  say  that  changes  are 
going  to  be  made  and  policies  will  be  followed  more  consistently,  but 
do  they  actually  follow  up  on  their  word  once  the  storm  has  passed? 

Do  not  say  something  just  to  look  good  in  front  of  the  public;  they  will 
know  if  you  mean  it  or  not.  Give  weekly  updates  and  use  the  power 
of  social  media  to  communicate  to  consumers  about  the  progress  your 
company  makes  as  it  works  toward  a  solution.  If  100  products  have 
been  tested,  let  the  public  know.  There  are  enough  resources  available 
today  to  control  the  media  and  communicate  a  company’s  commitment 
to  its  consumers.  It  is  never  more  important  than  in  a  time  of  crisis  to 
communicate  and  reassure  the  public  that  things  will  be  all  right. 

Thoughts  on  a  Lessons  Learned  Program  at  Brigade  Level 
and  Below  for  Military  Units 

Generally,  a  brigade-size  military  organization  (3,000  to  4,500  Soldiers)  is 
the  first  level  within  the  military  that  would  be  appropriate  for  establishing 
an  LL  capability.  At  this  level,  there  are  no  dedicated  individuals  to  perform 
the  LL  mission.  It  would  be  necessary  to  assign  this  responsibility  to  a 
person  as  an  additional  duty.  These  duties  would  generally  reside  in  the 
operations  section  of  the  staff. 

In  terms  of  specific  functions,  you  would  more  than  likely  focus  on  the 
collect-analyze-share  functions.  Collection  might  be  as  simple  as  directing 
subordinate  organizations  or  battalions  to  provide  after  action  reports 
(AARs)  at  the  completion  of  each  mission  or  operation.  This  could  be 
directed  in  the  operation  order  or  it  could  be  codified  in  the  unit’s  tactical 
standing  operating  procedures.  The  format  may  be  similar  to  the  one  at 
Appendix  C  or  more  simplified. 

The  observation-discussion-recommendation  format  works  nicely  for  this 
purpose,  as  a  simplified  version.  Once  the  brigade  receives  the  AAR,  it  is 
provided  to  the  various  staff  sections  for  review  and  minimal  analysis.  If  the 
recommendations  are  deemed  appropriate  by  the  staff,  the  unit  commander 
can  elect  to  share  the  report  with  other  organizations  in  the  brigade.  A  copy 
should  also  be  provided  to  the  brigade’s  next  higher  headquarters  for  its 
assessment,  for  information,  and  for  further  dissemination.  The  goal  should 
be  to  rapidly  provide  the  observations  and  recommendations  or  lessons  to  as 
many  interested  units  as  possible  to  effect  change. 


47 


CENTER  FOR  ARMY  LESSONS  LEARNED 


Another  collection  technique  could  be  the  use  of  a  small  team  to  observe, 
with  permission,  a  training  event  or  exercise  conducted  by  another  unit 
in  your  organization.  Your  team  becomes  informal  observers  who  are  not 
directly  involved  in  the  event  but  benefiting  from  the  actions  or  inactions 
of  another  unit.  Once  you  gather  the  observations,  you  can  prepare  a  report 
or  briefing  to  inform  your  Soldiers  on  what  you  saw  and  the  best  way  to 
execute  specific  missions. 

In  most  instances,  analysis  of  the  observations  should  not  be  so  rigorous  that 
it  slows  the  sharing  of  information.  Rapid  sharing  of  information  remains 
paramount  to  the  success  and  effectiveness  of  your  program.  As  stated 
previously,  the  key  to  a  viable  program  is  honest  and  open  communication 
at  all  levels  and  a  work/training  environment  where  individuals  can  learn 
from  their  mistakes. 

The  following  is  a  small  sample  of  organizations  and  their  visions,  mission 
statements,  and  intents  that  dictate  their  LL  programs.  More  details  on  the 
agencies  and  their  methods  of  working  LL  are  covered  in  Appendix  D. 

U.S.  Army 

The  Army’s  mission  is  to  fight  and  win  our  nation’s  wars  by  providing 
prompt,  sustained  land  dominance  across  the  full  range  of  military 
operations  and  spectrum  of  conflict  in  support  of  combatant  commanders. 
The  Army  does  this  by  learning  from  its  experiences  and  documenting 
them  for  forward  units  to  use  on  a  daily  basis.  The  documentation  is  done 
through  the  Center  for  Army  Lessons  Learned  (CALL)  and  the  Joint 
Lessons  Learned  Information  System  (JLL1S).  CALL  collects,  analyzes, 
disseminates,  integrates,  and  archives  Army  and  joint,  interagency, 
intergovernmental,  and  multinational  (JIIM)  observations,  insights,  lessons 
(OIL),  and  tactics,  techniques,  and  procedures  (TTP)  to  support  full- 
spectrum  military  operations.  One  of  the  most  important  operative  tasks  in 
CALL’S  mission  statement  is  collection.  Collection  of  the  latest  OIL  and 
TTP  and  their  subsequent  integration  into  the  operational  and  institutional 
Army  helps  units  and  Soldiers  meet  the  serious  challenges  posed  by  today’s 
operating  environment.  CALL  collects  OIL,  TTP,  and  operational  products 
and  records  from  the  field  primarily  through  five  methods: 

•  OIL  from  the  operational  Army. 

•  AARs. 

•  CALL  liaison  officers. 

•  CALL  collection  and  analysis  teams. 

•  Operational  products  and  records  submitted  from  the  operational 
Army. 
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Department  of  Energy  (DOE) 

DOE’s  overarching  mission  is  to  advance  the  national,  economic,  and 
energy  security  of  the  United  States;  to  promote  scientific  and  technological 
innovation  in  support  of  that  mission;  and  to  ensure  the  environmental 
cleanup  of  the  national  nuclear  weapons  complex.  The  department’s 
strategic  goals  to  achieve  the  mission  are  designed  to  deliver  results  along 
five  strategic  themes: 

•  Energy  Security:  Promoting  America’s  energy  security  through 
reliable,  clean,  and  affordable  energy. 

•  Nuclear  Security:  Ensuring  America’s  nuclear  security. 

•  Scientific  Discovery  and  Innovation:  Strengthening  U.S.  scientific 
discovery,  economic  competitiveness,  and  improving  quality  of  life 
through  innovations  in  science  and  technology. 

•  Environmental  Responsibility:  Protecting  the  environment  by 
providing  a  responsible  resolution  to  the  environmental  legacy  of 
nuclear  weapons  production. 

•  Management  Excellence:  Enabling  the  mission  through  sound 
management. 

Within  these  themes  are  16  strategic  goals  that  are  designed  to  help  DOE 
successfully  achieve  its  mission  and  vision.  One  of  these  goals  is  to  develop 
an  LL  center. 

The  DOE  corporate  LL  database  provides  a  central  clearinghouse  that 
allows  ready  access  to  and  communication  about  collected  information  on 
a  timely,  unimpeded  basis  by  all  DOE  elements.  The  database  is  used  to 
collect  and  share  LL  and  best  practices  pertaining  to  all  DOE  activities. 

DOE  corporate  LL  is  a  Web-based  LL  tool  designed  to  facilitate  the  sharing 
of  information  in  a  consistent  and  timely  manner  among  headquarters 
elements,  contractor,  and  subcontractor  entities.  The  DOE  LL  application 
provides  a  mechanism  for  communicating  experiences  throughout 
management  and  across  functional  areas.  The  sharing  of  LL  can  potentially 
reduce  risk,  improve  efficiency,  and  enhance  the  cost  effectiveness  of  DOE 
processes  and  operations.  DOE  LL  is  a  feedback  mechanism  for  the  DOE 
complex  intended  for  facility  management  use  and  promoting  continuous 
improvement  in  defining  and  planning  work.  It  can  be  used  to  identify 
LL  for  improving  performance,  planning,  and  for  correcting  hazardous 
conditions. 

DOE  LL  also  provides  an  LL  feedback  mechanism  for  the  job  planner  for 
selected  types  of  work.  DOE’s  integrated  safety  management,  feedback,  and 
improvement  function  encourages  the  use  of  LL  during  hazard  analysis  and 
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work-planning  activities.  DOE  LL  uses  an  integrated,  user-friendly,  Web- 
enabled  PC  browser  interface  capable  of  generating  topical  LL  extracted 
from  the  DOE  LL  database  and  external  sites.  The  output  generated  is  based 
on  previously  established  user  profiles  to  provide  customized  LL  reports. 
The  data  in  DOE  LL  is  updated  daily  and  compiled  after  SME  review  of 
DOE  LL  reports  submitted  from  across  the  DOE  complex. 

National  Aeronautics  and  Space  Administration  (NASA) 

NASA’s  mission  is  to  pioneer  the  future  in  space  exploration,  scientific 
discovery,  and  aeronautics  research.  To  do  that,  thousands  of  people  have 
been  working  around  the  world  —  and  off  of  it  —  for  50  years  trying  to 
answer  some  basic  questions.  What’s  out  there  in  space?  How  do  we  get 
there?  What  will  we  find?  What  can  we  learn  there,  or  learn  just  by  trying  to 
get  there,  that  will  make  life  better  here  on  Earth? 

NASA  Procedural  Requirement  (NPR) 

The  NPR  establishes  the  agency’s  requirements  for  collecting,  assessing, 
validating,  documenting,  and  infusing  LL  recommendations  involving  but 
not  limited  to  engineering,  technical,  science,  operations,  administrative, 
procurement,  management,  safety,  maintenance,  training,  flight  or  ground- 
based  systems,  facilities,  medical,  and  other  activities.  The  center-level 
LL  committees  are  the  key  organizational  elements  in  administering  this 
process.  An  agency-level  LL  steering  committee  facilitates  knowledge 
sharing  of  LL  activities  across  NASA  centers. 

The  LL  process  is  a  two-level  (centers  and  headquarters)  set  of  information 
management  processes  designed  to  preserve  institutional  knowledge, 
communicate  experiences  that  can  potentially  reduce  risk,  improve 
efficiency,  promote  validated  practices,  and/or  improve  performance  in 
the  areas  identified  above.  Lessons  are  collected  from  individuals,  projects 
and  programs,  or  supporting  organizations  primarily  at  the  center  level. 

The  content  of  LL  systems  in  the  NASA  environment  are  discoverable 
and  searchable  across  the  agency  to  the  broadest  extent  possible.  Lesson 
recommendations  are  assessed  for  potential  changes  to  policy,  procedures, 
guidelines,  technical  standards,  training,  education  curricula,  etc.  and 
infused  back  into  the  system  via  existing  corrective  action  systems.  NASA’s 
process  is  to  capture  the  knowledge  and  the  other  referenced  documents 
through  two  principal  requirements:  (1)  establishment  of  LL  committees 
at  the  center  level  and  (2)  closed-loop  infusion  of  LL  recommendations 
into  center  and  headquarters  documentation  and  training.  Contractors  are 
encouraged  to  use  their  existing  LL  processes  and  systems  where  they  meet 
the  requirements  of  the  NPR. 
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Appendix  A 

Sample  Detailed  Collection  Plan  Format 

Background.  What  is  the  situation  this  collection  plan  is  covering? 

Example:  On  12  January  2010,  the  Caribbean  island  of  Hispaniola  was 
rocked  by  a  7.2  magnitude  earthquake.  The  epicenter  of  the  quake  was  just 
south  of  the  Haitian  capital  city  of  Port  au  Prince,  home  to  approximately 
two  million  of  the  roughly  nine  million  people  in  the  country.  Devastation 
to  the  country  was  nearly  total.  Within  minutes,  tens  of  thousands  of 
people  were  dead  and  hundreds  of  thousands  were  left  homeless.  So  badly 
damaged  was  Haiti’s  government  and  infrastructure  that  it  was  virtually 
ineffectual  and  unable  to  adequately  respond.  It  was  one  of  the  Western 
Hemisphere’s  most  significant  natural  disasters  in  recent  history.  The  U.S. 
government  (USG)  relief  has  been  named  Operation  Unified  Response. 

Purpose.  Why  are  you  doing  this? 

Example:  In  its  broadest  sense,  the  purpose  of  the  study  is  to  collect 
best  practices  and  lessons  associated  with  the  response  by  the  USG  and 
international  community  to  the  Haiti  earthquake  disaster.  In  particular,  the 
study  will  examine  Department  of  Defense  (DOD)  and  USG  interagency 
(IA)  actions  associated  with  Operation  Unified  Response  in  light  of 
key  lessons  learned  (LL)  from  USG  participation  in  past  international 
humanitarian  assistance  disaster  relief  (HADR)  operations.  This  is  to 
identify  common  themes  that  inform  doctrine  organization,  training, 
materiel,  leadership  and  education,  personnel,  and  facilities  (DOTMLPF) 
solutions  that  can  be  applied  to  future  USG  HADR  endeavors. 

Key  Tasks.  What  are  the  tasks  necessary  for  this  collection  to  make  this 
happen?  Who  will  do  them? 

Examples: 

•  Research  Division:  Research  past  flood  after  action  reports  (AARs) 
and  provide  links  and/or  copies  on  the  shared  drive. 

•  Collection  Division:  Chair  weekly  meetings  to  review  the  past  weeks’ 
activities  to  determine  what  observations  and  lessons  may  have 
developed  for  rapid  sharing. 

End  State.  What  will  this  effort  produce? 

Example:  Referencing  documented  LL  from  USG  participation  in  past 
HADR  operations,  identify  challenges  and  issues  associated  with  DOD 
and  USG  1A  execution  of  Operation  Unified  Response,  and  identify 
and  document  applicable  LL  and  best  practices.  The  team  will  provide 
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actionable  recommendations  through  briefings  and  supplemental  written 
products  as  required. 

Scope.  What  are  the  limits  of  this  collection  effort? 

Example:  Per  its  charter  as  codified  in  Chairman  of  the  Joint  Chiefs  of 
Staff  Instruction  3150. 25D,  the  Joint  Center  for  Operational  Analysis 
(JCOA)  will  focus  collection  primarily  at  the  joint  and  operational  levels. 
Specifically,  activities  and  issues  related  to  the  Headquarters,  U.S.  Southern 
Command  (HQ  USSOUTHCOM)  and  its  subordinate  joint  task  force 
(JTF)  component  commands  and  supporting  service  entities/organizations 
involved  in  the  operation,  as  applicable.  Additionally,  collection  will  focus 
on  aspects  of  DOD  support  to  other  USG  entities,  in  particular  combatant 
commands,  and  service  support  to  the  Department  of  State  (DOS)  and  the 
U.S.  Agency  for  International  Development  (USAID),  where  there  are  clear 
lessons  that  either  impact  considerations  for  joint  DOTMLPF  or  impact  the 
manner  in  which  the  USG  participates  in  future  HADR  operations  of  this 
magnitude. 

Concept.  How  will  this  work  together?  Are  there  specific  areas  that  need  to 
be  looked  at  based  on  past  experiences  and  lessons? 

Example:  Using  U.S.  Joint  Forces  Command  resources  and  coordinating 
with  DOS,  USAID,  and  other  LL  organizations,  JCOA  will  conduct  an 
in-stride  study  on  the  international  HADR  response  following  the  Haiti 
earthquake,  documenting  challenges  and  best  practices.  JCOA  will  also 
serve  as  a  “directed  telescope,”  focusing  on  the  specific  issues  the  JTF  and 
USSOUTHCOM  leadership  believe  are  most  useful  for  meaningful  outputs 
and  for  informing  decision  makers  on  issues  they  need  to  consider.  JCOA 
will  provide  initial  and  ongoing  feedback  on  the  evolution  of  the  crisis,  the 
changing  tasks,  and  force  requirements  over  time  based  on  similar  incidents 
—  anticipating  emerging  challenges  and  possible  complications. 

Hypotheses  and  Related  Questions.  The  hypothesis  is  a  statement  of  what 
you  are  trying  to  confirm  or  deny  for  each  issue  in  the  collection  plan.  You 
will  probably  have  multiple  issues.  List  the  questions  you  want  to  ask  for 
each  issue.  They  can  be  organized  or  grouped  by  staff  function  or  by  any 
other  categorization  process  the  collectors  want  to  use.  This  is  the  most 
important  section  of  the  collection  plan.  You  should  never  answer  a  good 
question  with  a  yes  or  no. 

Example: 

Speed  of  Response:  The  speed  of  response  in  moving  people,  equipment, 
and  goods  at  the  onset  of  a  crisis  is  the  most  critical  element  of  successful 
HADR  operations. 
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•  How  would  you  rate  the  speed  of  response  by  the  United  States  to  this 
crisis? 

•  How  would  you  compare  the  speed  of  U.S.  response  to  other  relief 
providers? 

•  What  were  the  major  enablers  to  a  quick  response? 

•  What  were  the  major  challenges  to  a  quick  response? 

•  How  could  the  United  States  have  better  responded  to  this  crisis? 

Methodology.  How  will  this  effort  be  organized? 

Example:  The  collection  team  will  organize  to  cover  three  distinct  areas 
and  purposes:  JCOA  reach  back  (Suffolk,  VA);  JCOA  forward  (HQ, 
USSOUTHCOM);  and  JCOA  deployed  in  the  joint  operational  area.  The 
JCOA  reach  back  team  will  be  responsible  for  managing  incoming  data 
from  forward/deployed  team  members,  building  the  base  briefing  product, 
and  conducting  external  coordination/collaboration,  as  required.  The  JCOA 
forward  team  will  maintain  continuity  with  current  operational  concerns/ 
considerations  and  focus  activities  on  data  collection  —  conducting 
interviews  with  key  staff  leaders  to  support  the  study  —  and  provide 
feedback  to  the  USSOUTHCOM  staff.  The  JCOA  deployed  team  will  focus 
activities  on  data  collection  —  conducting  interviews  with  key  leaders 
from  DOD,  international  agencies,  nongovernmental  organizations,  and 
private  organizations  to  support  the  study  —  and  provide  feedback  to  the 
USSOUTHCOM  staff. 

Data  Collection  Procedures.  What  are  you  collecting  and  how? 

Example:  The  key  resources  used  to  evaluate  the  study  hypotheses  are 
the  data  and  interviews  collected  during  the  research.  JCOA  analysts  will 
form  data  collection  teams.  Each  team  will  be  required  to  address  multiple 
lines  of  activity  as  it  collects  quantitative  and  qualitative  data  and  conducts 
interviews  of  key  personnel  at  various  locations. 

The  following  documents  are  typical  of  those  needed  to  support  the  study. 
The  list  is  not  comprehensive,  and  other  documents  and  data  may  be 
discovered  by  the  deployed  teams: 

•  Quantitative  data  sources: 

o  Cables. 

o  Mission  reports/debriefs, 
o  Significant  activity  reports. 
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o  Operation  orders/fragmentary  orders, 
o  Standing  operating  procedures. 

•  Organization  charts. 

•  Daily  update  briefings. 

•  Unit  AARs. 

•  Memorandums. 

•  Briefings. 

•  Qualitative  data  sources: 

o  Subject  matter  expert  input. 

o  Observations  from  meetings,  conferences,  and  informal 
discussions. 

o  Interview  summaries  and  transcriptions  (vignettes  to  support  “the 
story”). 

Roster  of  Key  Personnel  and  Organizations 

Example:  Interviews  of  key  personnel  at  all  echelons  involved  with  Haiti 
and  other  related  operations  will  provide  the  key  insights  and  professional 
opinions  needed  to  validate  the  study  hypotheses.  Develop  a  list  of  key 
personnel  and  organizations  to  be  interviewed  and/or  observed  during  the 
study.  The  list  is  not  all  inclusive;  other  personnel  may  be  identified  to 
provide  details  pertinent  to  the  study. 

Data  Management  Procedures.  How  do  you  manage  collected  data?  File 
name  conventions?  Location?  Who  has  access?  Who  has  release  authority? 
What  are  the  classification  procedures? 
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Appendix  B 

Oral  Interview  Techniques 

General 

The  most  typical  way  to  gather  information  during  a  collection  effort 
is  through  the  oral  interview  process.  In  contemporary  knowledge 
management  terms,  oral  interviews  capture  what  often  remains  as  “tacit” 
or  silent  knowledge,  retained  and  used  only  by  that  individual,  group 
of  individuals,  or  unit  so  that  others  can  benefit  from  that  knowledge  or 
experience  too.  There  are  three  types  of  interviews:  structured,  semi- 
structured,  and  unstructured.  Each  of  these  have  associated  advantages  and 
disadvantages,  as  highlighted  below: 


Type 

Advantages 

Disadvantages 

Structured 

High  control. 

Minimal  variability. 
Question  response  analysis 
possible. 

Easier  to  estimate  duration. 
Easily  managed. 

Scripted. 

Little  to  no  opportunity 
for  discovery. 

Only  get  answers  to  what 
was  asked. 

Semi-Structured 

Topic/Issue  consistency. 
Opportunities  for 
deepening. 

Opportunities  for 
discovery. 

Topic  response  analysis 
possible. 

More  comfortable/relaxed. 

Less  controlled; 
introduces  more 
variability  since  questions 
may  not  be  identical. 
Requires  more  focus  by 
interviewer  to  manage/ 
guide  direction. 

Requires  time 
management. 

Unstructured 

Can  elicit  completely 
unanticipated  information. 
No  constraints;  anything 
is  OK. 

Extremely  casual. 

Requires  little 
management. 

Comparative  analysis  is 
difficult. 

Least  consistent  (topics/ 
areas). 

In  most  instances,  the  structured  or  semi-structured  interview  is  the  type 
preferred.  Oral  interviews  have  “pros  and  cons”  that  are  good  to  understand: 


•  Pros: 

o  Captures  information  that  would  otherwise  not  be  saved. 
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o  Provides  background  information  on  given  topics, 
o  Provides  personal  insights. 

o  Provides  useful  anecdotes  and  illustrations  from  first-hand 
experience. 

•  Cons: 

o  Oral  interview  may  contain  personal  biases, 
o  Some  interviewees  may  be  unwilling  to  discuss  mistakes, 
o  The  limitation  of  human  memory  is  the  greatest  challenge. 

Interview  Rules  of  Engagement 

Interviews  should  be  conducted  by  a  two-person  team,  whenever  possible. 
For  key  interviews,  use  a  digital  recorder  with  interviewee  approval. 
Explain  who  you  are  up  front,  the  mission  of  your  organization,  and  the 
purpose  of  the  collection  effort.  Explain  that  you  are  there  to  identify  and 
collect  information  to  support  the  “why  it  happened”  based  on  factual 
examples  and  that  you  want  to  avoid  personal  opinions,  if  possible,  unless 
they  bear  directly  on  the  issue.  It  is  also  good  to  highlight  what  the  final 
product  of  the  collection  will  be  and  to  tell  the  interviewees  how  they  can 
receive  a  copy  of  the  observation  report.  Generally,  you  will  explain  to  the 
organization  that  it  will  be  able  to  review  a  draft  copy  of  the  report  before 
it  goes  to  final  print.  Remind  the  interviewees  that  you  are  not  an  evaluator 
or  inspector,  and  rules  of  nonattribution  will  be  in  effect  if  they  so  desire. 
You  are  there  to  get  them  to  “tell  their  story”  and  to  make  recommendations 
on  how  the  performance  of  units  or  organizations  that  follow  them  may  be 
improved  by  benefiting  from  their  experiences  and  lessons  learned  (LL). 
Tell  the  interviewees  you  do  not  conduct  interviews  “off  the  record,”  and 
thank  them  for  their  time  and  participation. 

Before  you  start  the  interview,  it  is  appropriate  to  ask  a  few  lead-in 
questions  that  are  not  part  of  the  collection  plan  but  are  designed  to  give 
the  interviewer  an  idea  of  the  qualifications  and  experience  level  of  the 
interviewee.  This  gives  you  an  idea  if  you  are  talking  to  the  right  subject 
matter  expert.  The  following  questions  are  examples  you  can  use: 

•  How  long  have  you  been  with  the  organization? 

•  How  long  have  you  had  this  current  job  or  position? 

•  How  familiar  are  you  with  your  personnel  and  with  applicable  policy 
or  doctrine  manuals? 
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•  What  type  of  training  did  you  receive  before  you  deployed  or  got  this 
job,  and  were  you  satisfied  with  it? 

Before  the  Interview 

•  Write  out  a  list  of  questions  beforehand.  This  is  your  collection  plan. 

•  Try  to  construct  the  interview  chronologically  or  by  some  other  logical 
construct. 

•  Contact  and  orient  the  interviewee  prior  to  the  interview. 

•  Provide  questions  to  the  interviewee  in  advance,  if  possible.  In  many 
instances,  military  units  will  request  a  full  list  of  the  questions  (the 
collection  plan)  that  will  be  asked  weeks  before  the  collection  starts. 

•  Make  sure  your  digital  recorder  is  functioning  properly  and  you  know 
how  to  use  it.  Carry  extra  batteries. 

•  Bring  notepaper  and  pens. 

•  Two  sets  of  ears  are  better  than  one,  so  take  a  team  member  with  you, 
if  possible. 

During  the  Interview 

•  Make  introductions. 

•  Explain  the  purpose  of  the  interview  and  the  collection  effort.  Ensure 
the  interviewee  understands  you  are  not  an  evaluator  or  inspector. 

•  Get  permission  to  digitally  record  the  interview. 

•  Turn  on  the  recorder,  state  your  name,  the  name  of  the  interviewee 
and  his  job  position,  and  the  date,  and  announce  that  the  interview 
is  unclassified.  Do  not  ask  for  personal  information,  such  as  the 
interviewee’s  social  security  number. 

•  If  possible,  use  two  digital  voice  recorders  at  the  same  time  for 
backup. 

•  Tell  the  interviewee  that  if  he  goes  into  a  classified  area,  he  must  state 
so  beforehand  so  you  can  turn  off  the  recorder. 

•  Ask  your  lead-in  questions  to  determine  the  experience  level  of  the 
interviewee,  and  then  start  with  your  prepared  questions. 

•  Take  notes,  but  try  to  keep  your  focus  on  the  interviewee.  If  two 
interviewers  are  present,  one  can  take  notes  and  the  other  can  give 
attention  to  the  interviewee. 
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•  Interject  new  questions  if  necessary  to  clarify  or  go  into  more  detail  on 
certain  points;  this  is  the  “art”  of  interviewing. 

•  Sixty  minutes  is  about  the  maximum  length  for  an  interview  at  one 
sitting. 

After  the  Interview 

Fully  write  out  the  observation  as  soon  as  possible  following  the  interview. 
This  is  very  important.  The  information  is  fresh  and  you  have  less  chance 
of  confusing  it  with  other,  possibly  conflicting,  information  at  a  later 
date.  It  is  best  if  you  can  write  the  full  observation  within  24  hours  of  the 
interview.  A  good  goal  is  to  write  it  the  evening  following  the  interview. 
After  conducting  several  interviews,  it  is  very  easy  to  confuse  sources  and 
recommendations.  If  you  have  the  time,  invite  the  interviewee  to  make 
revisions  or  clarifications  to  the  text,  point  out  confusing  passages,  and  ask 
if  you  have  correctly  stated  the  observation. 

Label  the  tape  with  name,  date,  and  location,  as  required.  If  there  is  a 
chance  an  observation  is  classified,  have  the  unit  security  officer  review 
it.  Make  the  necessary  changes  in  an  attempt  to  keep  the  observation 
unclassified. 

Thank  the  interviewee  for  his  time.  Ask  for  phone  numbers  and  e-mail 
addresses  and  if  you  can  contact  him  in  the  future  for  any  additional 
clarifications  once  the  observation  report  is  in  draft. 

Review  “due  outs”  from  or  to  the  interviewee.  Make  sure  you  follow 
through  by  providing  any  materials  or  information  requested  by  the 
interviewee  from  the  LL  organization’s  archives  or  databases  that  will  help 
him  do  his  job. 

Summary 

Be  prepared.  Always  read  up  on  the  subject  you  are  reporting  about  and  the 
person  you  are  interviewing.  Set  the  rules  for  the  interview  up  front.  Be  sure 
your  subject  understands  what  you  are  working  on  and  the  issues  you  are 
addressing.  Be  on  time.  The  worst  impression  you  can  make  is  being  late 
for  the  interview.  Be  polite  and  do  not  rush  your  interviewee.  It  is  important 
to  establish  rapport  and  a  level  of  comfort.  Listen  but  do  not  be  afraid  to 
interrupt  when  you  do  not  understand  a  point.  Try  to  maintain  eye  contact. 
This  will  make  the  interview  more  like  a  conversation  and  enable  everyone 
to  be  more  relaxed.  Finally,  review  your  notes  right  after  the  interview  and 
make  any  clarifications  necessary.  After  interviewing  several  people,  your 
notes  will  begin  to  “run  together”  if  you  do  not  have  some  way  to  draw 
distinctions  between  each  interview. 
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Begin  the  interview  by  reading  this  brief  introductory  statement.  (The 
introductory  statement  must  be  recorded.) 

This  is  (interviewer’s  name) _ 

The  date  is  (month,  day,  year) _ 

This  interview  is  with  (first  name  and  last  name;  spell  out  on  tape) 


who  has  served  as  (job  title) _ 

for  (name  of  organization/command) _ 

since 

(month/year) _ 

We  are  conducting  this  interview  at  (name) _ 

in  (city/state/country) _ 

This  interview  will  address  the  topic(s)  of  (list  major  topics  of 
discussion): _ 


The  purpose  of  this  interview  is  to  collect  information  based  on  needs, 
recommendations,  and  suggestions  that  can  be  used  to  improve  company 
capabilities.  The  information  will  be  used  to  support  management  in 
the  execution  of  responsibilities  to  organize,  train,  equip,  and  move 
toward  a  more  efficient  running  organization.  This  interview  may  be 
transcribed  and  posted  to  the  company  website  for  review  by  authorized 
individuals.  The  information  from  this  interview  may  be  made  available 
to  other  companies.  If  you  prefer,  we  can  conduct  the  interview  on  a 
nonattribution  basis,  meaning  the  interview  is  recorded  and  transcribed, 
but  identifying  information  is  removed  to  make  you  anonymous. 

Do  I  have  permission  to  record  this  interview  and  associate  your  name 

with  it?  (Subject  Response:  Yes/No) _ .  Your  candidness  during 

the  interview  is  appreciated,  but  understand  that  we  cannot  offer  legal 
immunity  for  information  you  disclose.  Do  you  have  any  questions 
before  we  start  the  interview?  (Subject  Response:  Yes/No) _ . 

Begin  interview  questions. 

Closing:  “Thank  you  for  your  participation.  This  concludes  the 
interview.” 


Figure  B-l.  Interview  outline 
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Interview  Summary  Worksheet 


Instructions: 

Indicate  whether  the  interview  is  to  be  transcribed  or  not  and  indicate 
the  priority.  Summaries  are  to  be  completed  by  the  interviewer.  After 
completion,  upload  this  form  to  the  company  website.  Send  an  e-mail  to 
the  transcriptionists  to  notify  them  that  the  interviews  and/or  summary 
worksheets  have  been  uploaded.  An  alternative  is  to  e-mail  the  summary 
worksheets  and/or  interviews  as  an  attachment. 


H  Interview  to  be  transcribed 

H  Interview  to  NOT  be 

transcribed  (due  to  poor  audio 

Priority  for  transcription: 

quality,  low  precedence,  in 

□  High 

written  form  only,  etc.) 

□  Medium 

EH  Low 

Identifying  information 

Date  of  interview: 

Location: 

Interviewer’s  name  (First,  MI,  Last): 
Subject’s  name  (First,  MI,  Last): 
Unit: _ 

Office: 

Primary  topics  of  discussion: 


Acronyms: 


Needs  statements: 


Figure  B-2.  Interview  summary  worksheet 
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Guidelines  for  Writing  a  Lessons  Learned  Report 

A  report  of  LL  should  address  some  key  issues: 

•  Assessment  of  goals  and  objectives. 

•  Identification  of  activities  or  areas  needing  additional  effort. 

•  Identification  of  effective  activities  or  strategies. 

•  Comparison  of  costs  and  results  of  different  activities. 

•  Assessment  of  the  roles  of  organizations  in  the  project  and  the 
interactions  among  the  organizations. 

To  assess  goals  and  objectives,  consider  these  questions: 

•  Were  the  program  objectives  appropriate  for  the  program  goals? 

•  Were  the  objectives  met? 

•  Does  any  new  information  about  the  issue  need  to  be  incorporated  into 
the  program  messages  or  design? 

To  determine  areas  where  additional  effort  is  needed,  consider  these 
questions: 

•  Were  any  objectives  unmet? 

•  Were  any  strategies  or  activities  unsuccessful? 

To  identify  effective  activities  or  strategies,  consider  these  questions: 

•  Were  some  objectives  met  as  a  result  of  successful  activities? 

•  Should  these  activities  be  continued,  renewed,  and  strengthened? 

•  Can  you  expand  these  activities  to  apply  to  other  audiences  or 
situations? 

To  compare  costs  and  results  of  different  activities,  consider  these 
questions: 

•  What  were  the  relative  costs  (including  staff  time)  and  results  of 
different  aspects  of  your  program? 

•  Did  some  activities  appear  to  work  as  well  as  others  but  cost  less? 
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To  assess  the  roles  of  organizations  in  the  project  and  the  way  these 
organizations  worked  together,  consider  these  questions: 

•  Did  any  conflicts  of  organizational  agendas  or  operating  styles  occur? 

•  How  did  the  timing  of  the  program  coordinate  with  the  different 
organizations  involved? 
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Appendix  C 

Military  After  Action  Reviews/Reports 

General 

One  of  the  most  important  collection  techniques  used  in  the  U.S.  Army  and 
many  other  joint  organizations  is  the  after  action  review/report  or  AAR.  The 
concept  of  the  AAR  can  be  easily  adapted  to  fit  anyone’s  lessons  learned 
(LL)  program,  whether  it  is  government  or  civilian.  However,  the  examples 
used  here  are  worded  to  support  a  military  organization. 

When  the  term  AAR  is  used,  it  can  mean  two  different  collection 
techniques;  however,  both  provide  very  important  observations  and  lessons 
to  a  military  unit,  its  higher  headquarters,  and  the  force  in  general.  The  two 
forms  of  AARs  are: 

•  After  action  review:  A  verbal,  professional  discussion  of  a  unit’s 
actions  that  typically  occurs  immediately  after  a  training  event, 
combat  operation,  or  other  mission  that  determines  what  should  have 
happened,  what  actually  happened,  what  worked,  what  did  not  work 
and  why,  and  the  key  procedures  a  unit  wants  to  sustain  or  improve. 

•  After  action  report:  A  written  report  that  is  typically  submitted 
after  a  training,  combat  operation,  or  other  mission  that  normally 
documents  a  unit’s  actions  for  historical  purposes  but  also  provides 
key  observations  and  LL.  Portions  of  this  document  are  very  similar  to 
an  observation  report. 

Within  the  U.S.  Army,  no  concept  is  given  more  credit  for  changing  the  way 
it  trains  or  fights  than  the  AAR  process.  AARs  help  provide  Soldiers  and 
units  feedback  on  mission  and  task  performances  in  training  and  in  combat. 
They  identify  how  to  correct  deficiencies,  sustain  strengths,  and  focus  on 
the  performance  of  specific  mission-essential  task  list  training  objectives. 
The  verbal  AAR  conducted  after  every  operation  played  a  major  role  in 
transforming  how  the  U.S  Army  trained  and  fought  after  the  Vietnam  War 
when  it  was  implemented  at  the  combat  training  centers  (CTCs).  That, 
coupled  with  the  completion  of  a  written  unit  AAR  after  the  completion  of 
operations,  is  the  major  driver  of  any  military  LL  program.  You  cannot  have 
an  effective  LL  program  without  the  AAR. 

The  following  pages  provide  sample  formats  that  are  examples  only  and 
can  be  easily  modified  to  meet  any  situation  or  mission.  Admittedly,  these 
examples  are  designed  for  tactical  military  formations.  However,  the  basic 
concepts  can  be  adapted  to  any  government  or  nongovernment  organization. 
The  main  point  is  that  any  good  AAR  must  provide  LL  to  ultimately 
improve  and  enhance  organizational  performance. 
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After  Action  Review  Format 
Introduction  and  Rules 

The  training  exercise  or  operation  is  over.  It  is  now  time  to  conduct  the 
AAR.  A  facilitator  for  the  AAR  should  be  designated.  For  training  events, 
the  AAR  facilitator  may  be  a  CTC  observer/trainer  who  controlled  the 
exercise.  In  combat,  the  AAR  facilitator  is  typically  the  unit  commander 
or  operations  officer.  The  leader  should  start  by  reviewing  the  purpose  and 
sequence  of  the  AAR  to  ensure  everyone  understands  what  an  AAR  is  and 
how  it  works.  His  introduction  should  include  the  following  thoughts: 

•  An  AAR  is  a  dynamic,  candid,  professional  discussion  of  training  that 
focuses  on  unit  performance  against  the  Army  standard  for  the  tasks 
being  trained.  Everyone  can,  and  should,  participate  if  they  have  an 
insight,  observation,  or  question  that  will  help  the  unit  identify  and 
correct  deficiencies  or  maintain  strengths. 

•  An  AAR  is  not  a  critique.  No  one,  regardless  of  rank,  position,  or 
strength  of  personality,  has  all  of  the  information  or  answers.  AARs 
maximize  training  benefits  by  allowing  Soldiers,  regardless  of  rank,  to 
learn  from  each  other. 

•  An  AAR  does  not  grade  success  or  failure.  There  are  always 
weaknesses  to  improve  and  strengths  to  sustain. 

Figure  C-l  contains  a  recommended  sequence  for  conducting  an  AAR. 

Soldier  participation  is  directly  related  to  the  atmosphere  created  during 
the  introduction.  The  AAR  leader  should  make  a  concerted  effort  to  draw 
in  and  include  Soldiers  who  seem  reluctant  to  participate.  The  following 
techniques  can  help  the  leader  create  an  atmosphere  conducive  to  maximum 
participation.  He  should — 

•  Enter  the  discussion  only  when  necessary. 

•  Reinforce  the  fact  that  it  is  permissible  to  disagree. 

•  Focus  on  learning,  and  encourage  people  to  give  honest  opinions. 

•  Use  open-ended  and  leading  questions  to  guide  the  discussion  of 
Soldier,  leader,  and  unit  performance. 

•  Appoint  a  note  taker. 

In  some  instances,  it  may  be  appropriate  to  separate  AARs  by  Soldier  rank 
to  get  candid  comments.  This  may  be  more  important  for  the  lower  enlisted 
ranks. 
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Sequence  for  Conducting  AARs 

•  Introduction  and  rules. 

•  Review  of  objectives  and  intent: 

o  Training  objectives. 

o  Commander’s  mission/intent  (what  was  supposed  to  happen), 
o  Opposing  force  (OPFOR)  commander’s  mission/intent. 

•  Summary  of  recent  events  (what  actually  happened). 

•  Discussion  of  key  issues: 

o  Chronological  order  of  events, 
o  Warfighting  functions  (WFFs). 
o  Key  events/themes/issues. 

•  Discussion  of  optional  issues: 

o  Tasks  to  sustain/improve, 
o  Fratricide, 
o  Soldier/leader  skills, 
o  Statistics. 

•  Discussion  of  force  protection  (safety). 

•  Closing  comments  (summary). 

Figure  C-l.  Sequence  for  conducting  AARs 

Review  of  Objectives  and  Intent 
Training  Objectives 

The  AAR  leader  should  review  unit  training  objectives  for  the  training 
mission(s)  the  AAR  will  cover.  In  combat,  he  should  review  the  objectives 
of  the  operation.  He  should  also  restate  the  tasks  being  reviewed  as  well  as 
the  conditions  and  standards  for  the  tasks. 

Commander’s  Mission  and  Intent  (What  Was  Supposed  to  Happen) 

Using  maps,  operational  graphics,  terrain  boards,  and  so  on,  the  commander 
should  restate  the  mission  and  his  intent.  Then,  if  necessary,  the  discussion 
leader  should  guide  the  discussion  to  ensure  everyone  understands  the 
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plan  and  the  commander’s  intent.  Another  technique  is  to  have  subordinate 
leaders  restate  the  mission  and  discuss  their  commander’s  intent. 

OP  FOR  Commander’s  Mission  and  Intent 

In  a  formal  AAR,  the  OPFOR  commander  explains  his  plan  to  defeat 
friendly  forces.  He  uses  the  same  training  aids  as  the  friendly  force 
commander  so  that  participants  can  understand  the  relationship  of  both 
plans.  In  actual  combat,  this  role  could  be  performed  by  the  S-2  or  someone 
knowledgeable  about  what  the  enemy  was  attempting  to  accomplish. 

Summary  of  Recent  Events  (What  Actually  Happened) 

The  AAR  leader  now  guides  the  review  using  a  logical  sequence  of 
events  to  describe  and  discuss  what  happened.  He  should  not  ask  yes 
or  no  questions,  but  encourage  participation  and  guide  discussion  by 
using  open-ended  and  leading  questions.  An  open-ended  question  has  no 
specific  answer  and  allows  the  person  answering  to  reply  based  on  what 
was  significant  to  him.  Open-ended  questions  are  also  much  less  likely  to 
put  the  person  on  the  defensive.  This  is  more  effective  in  finding  out  what 
happened.  For  example,  it  is  better  to  ask, 

“Sergeant  Johnson,  what  happened  when  your  Bradley  crested  the  hill?” 
rather  than — 

“Sergeant  Johnson,  why  didn’t  you  engage  the  enemy  tanks  to  your  front?” 

As  the  discussion  expands  and  more  Soldiers  add  their  perspectives, 
what  really  happened  will  become  clear.  Remember,  this  is  not  a  critique, 
evaluation,  or  lecture;  the  AAR  leader  does  not  tell  the  Soldiers  or  other 
leaders  what  was  good  or  bad.  However,  the  AAR  leader  must  ensure 
specific  issues  are  revealed,  both  positive  and  negative  in  nature.  Skillful 
guidance  of  the  discussion  will  ensure  the  AAR  does  not  gloss  over 
mistakes  or  unit  weaknesses. 

Discussion  of  Key  Issues 

The  AAR  is  a  problem-solving  process.  The  purpose  of  discussion  is  for 
participants  to  discover  strengths  and  weaknesses,  propose  solutions,  and 
adopt  a  course  of  action  to  correct  problems.  Leaders  can  organize  the 
discussion  using  one  of  the  three  techniques  in  the  following  paragraphs. 

Chronological  Order  of  Events 

This  technique  is  logical,  structured,  and  easy  to  understand.  It  follows  the 
flow  of  training  from  start  to  finish  and  allows  Soldiers  to  see  the  effects  of 
their  actions  on  other  units  and  events.  By  covering  actions  in  the  order  they 
took  place,  Soldiers  and  leaders  are  better  able  to  recall  what  happened. 
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WFFs 

To  focus  and  structure  the  AAR,  the  U.S  Army  sometimes  uses  the  six 
WFFs  (movement  and  maneuver,  intelligence,  fires,  protection,  mission 
command,  and  sustainment).  By  focusing  on  each  WFF  and  discussing  it 
across  all  phases  of  the  training  exercise,  participants  can  identify  systemic 
strengths  and  weaknesses.  This  technique  is  particularly  useful  in  training 
staff  sections  whose  duties  and  responsibilities  directly  relate  to  one  or 
more  WFF.  Flowever,  leaders  using  this  technique  must  be  careful  not  to 
lose  sight  of  the  big  picture.  They  must  not  get  into  long  discussions  about 
WFFs,  which  do  not  relate  to  mission  accomplishment. 

Key  Events/Themes/Issues 

A  key  events  discussion  focuses  on  critical  events  that  directly  support 
training  or  mission  objectives  the  chain  of  command  identified  before 
the  operation  began.  Keeping  a  tight  focus  on  these  events  prevents  the 
discussion  from  becoming  sidetracked  by  issues  that  do  not  relate  to  the 
objectives.  This  technique  is  particularly  effective  when  time  is  limited. 

One  of  the  strengths  of  the  AAR  format  is  its  flexibility.  The  leader  could 
use  the  chronological  format  to  structure  the  discussion;  then,  if  a  particular 
WFF  seems  to  have  systemic  issues  the  group  needs  to  address,  follow 
that  WFF  across  the  entire  exercise.  Once  that  topic  is  exhausted,  the  AAR 
could  proceed  using  the  chronological  format.  Each  technique  will  generate 
discussion  to  identify  unit  strengths,  weaknesses,  and  training  the  unit  needs 
to  improve  proficiency.  Flowever,  the  leader  must  remember  to: 

•  Be  specific,  avoiding  generalizations. 

•  Be  thorough. 

•  Not  dwell  on  issues  unrelated  to  mission  accomplishment. 

•  Focus  on  actions. 

•  Relate  performance  to  the  accomplishment  of  training  objectives. 

•  Identify  corrective  action  for  areas  of  weakness. 

•  Continually  summarize. 

Discussion  of  Optional  Issues 

In  addition  to  discussing  key  issues,  the  leader  might  also  address  several 
optional  topics,  included  in  the  following  paragraphs. 
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Tasks  to  Sustain/Improve 

This  technique  focuses  on  identifying  tasks  on  which  the  unit  is  proficient 
and  tasks  on  which  they  need  further  training.  The  intent  is  to  focus  training 
on  mission-essential  tasks  and  supporting  Soldier,  leader,  and  collective 
tasks  that  need  improvement  rather  than  training  to  known  strengths. 
Although  it  is  important  to  sustain  proficiency  on  tasks  whose  standards  the 
unit  has  met,  it  is  more  important  to  train  to  standard  on  new  or  deficient 
mission-essential  tasks.  Train  to  weakness,  not  to  strength. 

Fratricide 

All  incidents  or  near  incidents  of  fratricide,  whether  inflicted  by  direct  fire, 
indirect  fire,  or  close  air  support,  will  be  discussed  in  detail.  The  leader 
must  focus  on  identifying  the  cause  of  the  fratricide  and  develop  standing 
operating  procedures  (SOPs)  and  tactics,  techniques,  and  procedures 
(TTP)  to  prevent  it  in  the  future.  Regardless  of  the  environment  (training 
or  combat),  the  leader  must  swiftly  deal  with  all  fratricide  incidents.  As 
soon  as  possible  after  the  event,  an  AAR  should  be  held  to  discuss  the 
circumstances  surrounding  the  event,  using  the  following  discussion  points: 

•  How  and  why  did  the  incident  occur? 

•  How  were  friendly  personnel  and  equipment  identified? 

•  What  fire  control  measures  were  in  place  where  the  fratricide  occurred, 
and  how  effective  were  they? 

•  How  did  the  commander’s  risk  assessment  and  overall  intent  for  the 
mission  address  the  issue  of  fratricide? 

Soldier/Leader  Skills 

Through  discussion,  the  unit  can  identify  critical  Soldier  and  leader  skills 
that  affected  unit  or  individual  performance.  The  leader  should  note  these 
skills  for  retraining  or  for  future  unit  training.  (Often  it  is  best  to  discuss 
leader  skills  in  a  separate  meeting  or  AAR  specifically  for  that  purpose. 

This  allows  for  a  candid  discussion  of  leadership  issues  without  wasting 
unit  AAR  time  best  spent  on  reviewing  the  entire  training  exercise.)  The 
AAR  leader  for  follow-on  meetings  should  be  a  member  of  the  unit,  so 
participants  can  candidly  address  key  training  issues  without  fear  of  airing 
dirty  laundry  in  front  of  outsiders. 

Statistics 

Statistics  is  a  double-edged  sword.  Effective  feedback  requires  participants 
to  measure,  collect,  and  quantify  performance  during  the  training  exercise. 
Statistics  supply  objective  facts  that  reinforce  observations  of  both  strengths 
and  weaknesses.  The  danger  lies  in  statistics  for  statistics’  sake.  Chart 
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after  chart  of  ratios,  bar  graphs,  and  tables  quickly  obscures  any  meaning 
and  lends  itself  to  a  “grading”  of  unit  performance.  This  stifles  discussion 
and  degrades  the  AAR’s  value.  Statistics  and  statistics-based  charts 
should  identify  critical  trends  or  issues  and  reinforce  teaching  points.  An 
example  for  an  armored  unit  would  be  to  link  the  number  of  rounds  fired 
to  the  number  of  enemy  vehicles  destroyed.  This  would  provide  a  good 
indication  of  unit  gunnery  skills.  Judicious  use  of  statistic  feedback  supports 
observations  and  provides  a  focus  to  AAR  discussions. 

Discussion  of  Force  Protection  (Safety  Issues) 

Safety  is  every  Soldier’s  business  and  applies  to  everything  a  unit  does  in 
the  field  and  in  garrison.  Safety  should  be  specifically  addressed  in  every 
AAR  and  discussed  in  detail  when  it  impacts  unit  effectiveness  or  Soldier 
health.  The  important  thing  is  to  treat  safety  precautions  as  integral  parts  of 
every  operation. 

Closing  Comments  (Summary) 

During  the  summary,  the  AAR  leader  reviews  and  summarizes  key  points 
identified  during  the  discussion.  He  should  end  the  AAR  on  a  positive  note, 
linking  conclusions  to  future  training.  He  should  then  leave  the  immediate 
area  to  allow  unit  leaders  and  Soldiers  time  to  discuss  the  training  in  private. 

Written  After  Action  Report  Format 

The  template  below  (Figure  C-2)  serves  as  an  excellent  guide  to  what 
a  commander  may  elect  to  cover  in  his  unit’s  written  AAR.  The  AAR 
provides  TTP  and  LL  for  dissemination  to  the  Army.  For  example,  the 
Center  for  Army  Lessons  Learned  archives  all  AARs  it  receives  and  through 
its  website  makes  them  available  to  units  preparing  for  combat  operations  or 
training  events. 

The  AAR  can  be  organized  by  WFF  or  by  phases,  as  in  the  example.  It 
should  be  arranged  chronologically,  where  possible.  The  format  is  flexible; 
however,  two  key  purposes  of  the  written  AAR  shoidd  be  to  (1)  document 
the  operations  conducted  by  the  unit  for  historical  purposes  and  (2)  provide 
best  practices  and  lessons  in  the  observation-discussion-recommendation 
format  that  can  be  used  to  inform  the  Army’s  LL  program  (see  example  in 
Figure  C-3).  What  worked  well  should  receive  as  much  attention  as  what 
did  not. 
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1.  Report  cover  page: 

Classification. 

Preparing  headquarters  or  organization. 

Location  of  report  preparation. 

Date  of  preparation. 

AAR  title. 

Period  covered:  (date  to  date). 

2.  Preface  or  foreword  signed  by  the  commander. 

3.  Table  of  contents. 

4.  Executive  summary  and  chronology  of  significant  events: 

Briefly  summarize  operations  for  all  phases;  include  key  dates 
for  each  phase  starting  with  deployment  and  ending  with 
redeployment. 

Include  numbers  of  Soldiers  deployed. 

Summarize  task  organization. 

Summarize  casualty  information. 

Summarize  key  LL. 

What  was  the  single  greatest  success  and  the  single  greatest 
shortcoming  or  challenge  from  the  unit’s  perspective? 

5.  Detailed  task  organization,  include  any  significant  changes/dates  as 
appropriate: 

Wiring  diagram,  including  attached  units/elements  and  named  task 
forces. 

Relationship  to  higher  headquarters  and  list  of  subordinate 
elements. 

6.  Predeployment  phase  with  dates: 

Unit’s  training  focus. 

What  should  have  been  accomplished  during  predeployment  that 
was  not  accomplished? 
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What  was  helpful  to  know  when  planning  the  deployment?  What 
did  you  wish  that  you  had? 

Discuss  logistics  and  personnel  shortages,  if  appropriate. 

Discuss  planning  for  rear  detachment  operations. 

Discuss  predeployment  LL  in  the  observation-discussion- 
recommendation  format. 

7.  Deployment/reception,  staging,  onward  movement,  and  integration 
(RSOI)  with  dates: 

Summarize  deployment/RSOI  operations. 

Discuss  RSOI  LL  in  the  observation-discussion-recommendation 
format. 

8.  Operations  phase  with  dates: 

Summarize  tactical  and  nontactical  operations  (sometimes 
beneficial  to  do  this  by  staff  section  or  WFF). 

Include  unit  participation  in  named  operations. 

List  of  key  operation  orders  (OPORDs)  and  fragmentary  orders 
(FRAGOs). 

Discuss  operations  phase  LL  in  the  observation-discussion- 
recommendation  format  (sometimes  beneficial  to  address  by  WFF). 

9.  Relief  in  place/transfer  of  authority  (RIP/TOA)  with  dates: 

Discuss  planning  and  overlap. 

List  or  discuss  key  discussion  topics  between  outgoing  and 
incoming  organizations. 

Include  (either  here  or  as  an  appendix)  any  SOPs,  TTP,  or 
checklists. 

Discuss  RIP/TOA  LL  in  the  observation-discussion- 
recommendation  format. 

10.  Redeployment  activities  with  dates: 

Summarize  redeployment  activities  and  highlight  planning 
guidance  either  developed  or  received  from  higher  headquarters. 
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How  long  did  redeployment  take?  What  was  salvaged  or  destroyed 
(battle  losses  and  personnel  losses  —  killed  in  action,  wounded 
in  action,  missing  in  action,  and  Soldiers  classified  as  duty  status 
whereabouts  unknown)  during  the  unit’s  time  in  theater? 

Include  (either  here  or  as  an  appendix)  any  list  of  instructions,  TTP, 
or  checklists  developed. 

Discuss  redeployment  LL  in  the  observation-discussion- 
recommendation  format. 

11.  Post-deployment  activities: 

Discuss  combat  stress  planning  and  reintegration  activities. 

Discuss  plans  and  priorities  used  in  reconstituting  and  resetting  the 
unit. 

Discuss  Family  Support  Group  operations. 

Discuss  post-deployment  LL  in  the  observation-discussion- 
recommendation  format. 

12.  Provide  an  index/listing  of  all  mid-tour  and  final  unit  AAR  products, 
significant  command  briefings,  or  reports  published  separately: 

Include  classification,  titles,  and  distribution/disposition  of  reports. 

Include  a  staff  point  of  contact  or  section  for  follow-up 
coordination. 

13.  Distribution  (of  this  report). 

14.  Appendices  (as  appropriate): 

List  of  each  named  operation  or  major  event  with  dates. 

Applicable  maps. 

Photographs. 

Copies  of  key  OPORDs  and  FRAGOs. 

Particularly  useful  TTP  or  unit  products  developed. 

Predeployment  site  survey  information. 

Rear  detachment  operations. 

Unit  daily  journals. 

Figure  C-2.  Sample  written  AAR  format 
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Observation:  Media-on-the-battlefield  training  was  poorly  conceived  and 
planned. 

Discussion:  The  role-player  journalists  were  not  sufficiently  trained  for 
the  task  they  were  to  perform,  and  they  were  not  resourced  properly.  They 
did  not  know  the  scenario  and  asked  unrealistic  questions.  They  did  not 
provide  a  realistic  representation  of  a  journalist  on  the  battlefield  from  a 
major  news  organization. 

Recommendation:  You  must  train  role  players  for  media-on-the- 
battlefield  scenarios.  Role  players  should  be  given  a  character  description 
so  they  can  act  the  part.  They  should  understand  the  media  credential 
system,  ground  rules,  and  the  scenario  in  general.  They  should  be  capable 
of  engaging  in  a  dialogue  to  determine  the  essential  elements  of  a  news 
story. 

Figure  C-3.  Sample  observation-discussion-recommendation  format 


“You  must  learn  from  the  mistakes  of  others.  You  can’t  possibly  live  long 
enough  to  make  them  all  yourself.” 

—  Sam  Levenson 
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Appendix  D 

Government  Agencies  and 
Lessons  Learned  Centers 

Center  for  Army  Lessons  Learned 

The  Center  for  Army  Lessons  Learned  (CALL)  rapidly  collects,  analyzes, 
disseminates,  integrates,  and  archives  observations,  insights,  and  lessons 
(OIL);  tactics,  techniques,  and  procedures  (TTP);  and  operational  records 
to  facilitate  rapid  adaptation  initiatives  and  conduct  focused  knowledge 
sharing  and  transfer  that  informs  the  Army  and  enables  operationally  based 
decision  making,  integration,  and  innovation  throughout  the  Army  and 
within  the  interagency,  intergovernmental,  and  midtinational  environment. 

CALL  collects  OIL,  TTP,  and  operational  products  and  records  from  the 
field  primarily  through  five  methods: 

•  OIL  from  the  operational  Army. 

•  After  action  reports  (AARs). 

•  CALL  theater  observation  detachments. 

•  CALL  collection  and  analysis  teams. 

•  Operational  products  and  records  submitted  from  the  operational 
Army. 


Figure  D-l.  CALL  lessons  learned  (LL)  process 
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Figure  D-2.  CALL  deliberate  LL  process 

Joint  Analysis  and  Lessons  Learned  Centre  (JALLC) 

The  Joint  Analysis  and  Lessons  Learned  Centre  (JALLC)  is  NATO’s 
center  for  performing  joint  analysis  of  operations,  training,  exercises,  and 
concept  development  and  experimentation  collective  experiments,  including 
establishing  and  maintaining  an  interactive  managed  LL  database.  JALLC 
also  hosts  and  manages  NATO’s  LL  database,  where  lessons  are  captured, 
stored,  and  processed. 

JALLC ’s  Internet  LL  database  is  a  tool  to  coordinate  the  staffing  of 
NATO  operational  and  exercise  lessons  in  a  central,  accessible  location 
and  to  archive  these  lessons.  The  database  comprises  observations, 
lessons  identified  (LI),  and  LL.  Observations  are  detailed  examinations 
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of  phenomena  prior  to  analysis,  diagnosis,  or  interpretation.  LI  are 
observations  that  have  been  staffed  and  deemed  beneficial  to  others.  The 
staffing  includes  discussion  of  the  nature  and  causes  of  the  observation, 
recommendations  of  actions  to  be  taken,  and  a  proposed  action  body. 

LL  comprise  an  implemented  recommendation  action  that  produced  an 
improved  performance  or  increased  capability.  This  data  is  then  submitted 
by  users  from  across  NATO  and  are  refined  and  managed  by  JALLC,  which 
analyzes  the  information  and  forwards  it  to  appropriate  commands  for  the 
coordination  of  remedial  action. 


“Simplified  Further” 
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Figure  D-4.  NATO  LL  process  (simplified) 


NASA 

The  LL  process  is  a  two-level  (centers  and  headquarters)  set  of  information 
management  processes  designed  to  preserve  institutional  knowledge, 
communicate  experiences  that  can  potentially  reduce  risk,  improve 
efficiency,  promote  validated  practices,  and/or  improve  performance  in 
the  areas  identified  above.  Lessons  are  collected  from  individuals,  projects 
and  programs,  or  supporting  organizations,  primarily  at  the  center  level. 
The  content  of  LL  systems  in  the  NASA  environment  are  discoverable 
and  searchable  across  the  agency  to  the  broadest  extent  possible.  Lesson 
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recommendations  are  assessed  for  potential  changes  to  policy,  procedures, 
guidelines,  technical  standards,  training,  education  curricula,  etc.  and 
infused  back  into  the  system  via  existing  corrective  action  systems. 
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Figure  D-5.  NASA  LL  process  flow  diagram 

Department  of  Energy 

The  LL  program  facilitates  continuous  and  systematic  information 
sharing  and  learning  across  the  Department  of  Energy  (DOE)  complex. 

Each  DOE  employee,  contractor,  and  subcontractor  is  a  valuable  source 
of  knowledge,  information,  and  learning  that  can  be  tapped  to  provide 
enormous  benefits  —  cost  savings,  improved  safety,  greater  productivity, 
and  better  results.  These  benefits  can  be  multiplied  across  the  complex  if 
information  is  effectively  shared  and  employees  are  committed  to  using  the 
LL  information.  One  of  the  primary  goals  of  the  LL  program  is,  therefore,  to 
link  these  initiatives  to  make  accessing  and  sharing  information  across  sites 
and  programs  easy. 
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Figure  D-6.  DOE  LL  process 


Air  Force  (A9L) 

The  Air  Force  LL  program  exists  to  enhance  readiness  and  improve  combat 
capability  by  capitalizing  on  the  experiences  of  Airmen.  An  LL  is  defined 
as  an  insight  gained  that  improves  military  operations  or  activities  at  the 
strategic,  operational,  or  tactical  level  and  results  in  long-term,  internalized 
change  to  an  individual,  group  of  individuals,  or  an  organization.  Past 
experiences  also  assist  senior  leaders  in  programming,  budgeting,  and 
allocating  resources  to  make  changes  to  doctrine,  organizations,  training, 
materiel,  leadership  and  education,  personnel,  and  facilities.  An  LL  is  not  a 
compliance  “report  card,”  nor  is  it  automatically  accepted  and  implemented 
without  the  scrutiny  of  warfighters  and  functional  experts.  An  LL  is  not 
“owned”  by  any  one  organization. 
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Figure  D-7.  Air  Force  LL  process 


Marine  Corps 

The  Marine  Corps  Center  for  Lessons  Learned  (MCCLL)  collects  and 
analyzes  information  gained  by  Marines  in  operational  experiences  to 
produce  reports  and  recommendations  advancing  the  Marine  Corps 
reservoir  of  knowledge  and  stimulating  changes  in  the  way  the  Corps 
organizes,  trains,  equips,  and  sustains  Marines  in  combat. 

MCCLL  uses  the  collections  process  to  gather  information  for  follow-on 
analysis  and  drafting  of  MCCLL  collection  reports.  Formal  collections  by 
MCCLL  collection  teams  use  interviews,  questionnaires,  and  surveys. 

At  a  minimum,  a  good  collection  plan  will  state: 

•  What  you  want  to  know. 

•  Who  you  are  going  to  ask. 

•  Why  you  want  the  information. 

•  What  you  intend  to  do  with  the  information. 

•  How  you  will  collect  the  information. 
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Figure  D-8.  MCCLL  collection,  analysis,  report  process 

Lessons  Learned  Centers 

U.S.  Army 

Center  for  Army  Lessons  Learned 
10  Meade  Ave.  (Bldg.  50) 

Fort  Leavenworth,  KS  66027 
<http://usacac.army.mil/cac2/call/index.asp> 

U.S.  Air  Force  (A9L) 

Office  of  Air  Force  Lessons  Learned 
1777  N  Kent  St.,  Floor  6 
Rosslyn,  VA  22209-2 110 
<https://www.jllis.mil/USAF/> 

U.S.  Marine  Corps 

CG,  TECOM  (MCCLL) 

1019  Elliot  Road 
Quantico,  VA  22134 
<http://www.mccll.usmc.mil/> 
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U.S.  Navy 

1530  Gilbert  St.,  Suite  2128 
Norfolk,  VA  2351 1-2730 

Department  of  Energy 

U.S.  Department  of  Energy 

Office  of  NEPA  Policy  and  Compliance  (GC-54) 

1000  Independence  Ave.,  S.W. 

Washington,  DC  20585 

<http://nepa.energy.gov/lessons_learned.htm> 

U.S.  Special  Operations  Command 

HQ  Special  Operations  Command 
7701  Tampa  Point  Blvd 
MacDill  AFB,  FL  33621-5323 
<https://www.jllis.mil/ussocom/index.cfm> 

Canada 

Assistant  Deputy  Minister  (Public  Affairs) 
Department  of  National  Defence 
National  Defence  Headquarters 
Major-General  George  R.  Pearkes  Building 
101  Colonel  By  Drive 
Ottawa,  Ontario,  Canada 
K1A0K2 

NASA 

NASA  Headquarters 
Suite  5K39 

Washington,  DC  20546-0001 
<http://llis.nasa.gov/offices/oce/llis/home/> 

JALLC 

Avenida  Tenente  Martins  -  Monsanto 

1500-589  Lisbon 

Portugal 

Center  for  Complex  Operations 

Fort  Lesley  J.  McNair 
Washington,  DC  20319 
<http://ccoportal.org/> 
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Department  of  the  Interior 

1849  C  St.,  N.W. 

Washington  DC  20240 
<http://www.doi.gov/index.cfm> 

U.S.  Agency  for  International  Development 

U.S.  Agency  for  International  Development 
Ronald  Reagan  Building 
Washington,  D.C.  20523-1000 
<http://www.usaid.gov/> 

Department  of  Homeland  Security 

U.S.  Department  of  Homeland  Security 
Washington,  DC  20528 
<http://www.dhs.gov/index.shtm> 

EMP  Solutions 

<http://www.enterprisepmsolutions.com/> 
Society  for  Effective  Lessons  Learned  Sharing 
<http://tis.eh.doe.gov/ll> 

Federal  Emergency  Management  Agency 
<https://www.llis.dhs.gov/index.do> 

Federal  Bureau  of  Investigation 
<http://www.fbi.  gov/> 


83 


ESTABLISHING  A  LESSONS  LEARNED  PROGRAM 


PROVIDE  US  YOUR  INPUT 


To  help  you  access  information  quickly  and  efficiently,  the  Center  for  Army  Lessons  Learned 
(CALL)  posts  all  publications,  along  with  numerous  other  useful  products,  on  the  CALL 
website.  The  CALL  website  is  restricted  to  U.S.  government  and  allied  personnel. 


PROVIDE  FEEDBACK  OR  REQUEST  INFORMATION 


<http  ://call.ar  my.mil> 

If  you  have  any  comments,  suggestions,  or  requests  for  information  (RFIs),  use  the  following 
links  on  the  CALL  home  page:  “RFI  or  a  CALL  Product”  or  “Contact  CALL.” 


PROVIDE  OBSERVATIONS,  INSIGHTS,  AND  LESSONS  (OIL)  OR 
SUBMIT  AN  AFTER  ACTION  REVIEW  (AAR) 


If  your  unit  has  identified  lessons  learned  or  OIL  or  would  like  to  submit  an  AAR,  please 
contact  CALL  using  the  following  information: 


Telephone:  DSN  552-9569/9533;  Commercial  913-684-9569/9533 
Fax:  DSN  552-4387;  Commercial  913-684-4387 
NIPR  e-mail  address:  call.rflmanager@conus.army.mil 
SIPR  e-mail  address:  call.rflagent@conus.army.smil.mil 
Mailing  Address: 

Center  for  Army  Lessons  Learned 
ATTN:  OCC,  10  Meade  Ave.,  Bldg.  50 
Lort  Leavenworth,  KS  66027-1350 


TO  REQUEST  COPIES  OF  THIS  PUBLICATION 


If  you  would  like  copies  of  this  publication,  please  submit  your  request  at:  <http://call.army. 
mil>.  Use  the  “RFI  or  a  CALL  Product”  link.  Please  fill  in  all  the  information,  including  your 
unit  name  and  official  military  address.  Please  include  building  number  and  street  for  military 
posts. 
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PRODUCTS  AVAILABLE  “ONLINE” 


CENTER  FOR  ARMY  LESSONS  LEARNED 

Access  and  download  information  from  CALL’S  website.  CALL  also  offers  Web-based  access 
to  the  CALL  Archives.  The  CALL  home  page  address  is: 

<http://call.army.mil> 

CALL  produces  the  following  publications  on  a  variety  of  subjects: 

•  Combat  Training  Center  Bulletins,  Newsletters,  and  Trends 

•  Special  Editions 

•  News  Front  the  Front 

•  Training  Techniques 

•  Handbooks 

•  Initial  Impressions  Reports 

You  may  request  these  publications  by  using  the  “RFI  or  a  CALL  Product”  link  on  the  CALL 
home  page. 


COMBINED  ARMS  CENTER  (CAC) 
Additional  Publications  and  Resources 


The  CAC  home  page  address  is: 

<http://usacac.army.mil/cac2/index.asp> 

Center  for  Army  Leadership  (CAL) 

CAL  plans  and  programs  leadership  instruction,  doctrine,  and  research.  CAL  integrates  and 
synchronizes  the  Professional  Military  Education  Systems  and  Civilian  Education  System. 
Find  CAL  products  at  <http://usacac.army.mil/cac2/cal/index.asp>. 

Combat  Studies  Institute  (CSI) 

CSI  is  a  military  history  think  tank  that  produces  timely  and  relevant  military  history  and 
contemporary  operational  history.  Find  CSI  products  at  <http://usacac.army.mil/cac2/csi/ 
csipubs.asp>. 

Combined  Arms  Doctrine  Directorate  (CADD) 

CADD  develops,  writes,  and  updates  Army  doctrine  at  the  corps  and  division  level.  Find  the 
doctrinal  publications  at  either  the  Army  Publishing  Directorate  (APD)  <http://www.usapa. 
army.mil>  or  the  Reimer  Digital  Library  <http://www.adtdl.army.mil>. 

Foreign  Military  Studies  Office  (FMSO) 

FMSO  is  a  research  and  analysis  center  on  Fort  Leavenworth  under  the  TRADOC  G2.  FMSO 
manages  and  conducts  analytical  programs  focused  on  emerging  and  asymmetric  threats, 
regional  military  and  security  developments,  and  other  issues  that  define  evolving  operational 
environments  around  the  world.  Find  FMSO  products  at  <http://fmso.leavenworth.army.mil/>. 
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Military  Review  (MR) 

MR  is  a  revered  journal  that  provides  a  forum  for  original  thought  and  debate  on  the  art 
and  science  of  land  warfare  and  other  issues  of  current  interest  to  the  U.S.  Army  and  the 
Department  of  Defense.  Find  MR  at  <http://usacac.army.mil/cac2/militaryreview/index.asp>. 

TRADOC  Intelligence  Support  Activity  (TRISA) 

TRISA  is  a  field  agency  of  the  TRADOC  G2  and  a  tenant  organization  on  Fort  Leavenworth. 
TRISA  is  responsible  for  the  development  of  intelligence  products  to  support  the  policy¬ 
making,  training,  combat  development,  models,  and  simulations  arenas.  Find  TRISA  Threats  at 
<https://dcsint-threats. leavenworth.army.mil/default.aspx>  (requires  AKO  password  and  ID). 

Combined  Arms  Center-Capability  Development  Integration  Directorate  (CAC- 
CDID) 

CAC-CDIC  is  responsible  for  executing  the  capability  development  for  a  number  of  CAC 
proponent  areas,  such  as  Information  Operations,  Electronic  Warfare,  and  Computer  Network 
Operations,  among  others.  CAC-CDID  also  teaches  the  Functional  Area  30  (Information 
Operations)  qualification  course.  Find  CAC-CDID  at  <http://usacac.aiTny.mil/cac2/cdid/index. 
asp>. 

U.S.  Army  and  Marine  Corps  Counterinsurgency  (COIN)  Center 

The  U.S.  Army  and  Marine  Corps  COIN  Center  acts  as  an  advocate  and  integrator  for  COIN 
programs  throughout  the  combined,  joint,  and  interagency  arena.  Find  the  U.S.  Army/U.S. 
Marine  Corps  COIN  Center  at:  <http://usacac.army.mil/cac2/coin/index.asp>. 

Joint  Center  for  International  Security  Force  Assistance  (JCISFA) 

JCISFA’s  mission  is  to  capture  and  analyze  security  force  assistance  (SFA)  lessons  from 
contemporary  operations  to  advise  combatant  commands  and  military  departments  on 
appropriate  doctrine;  practices;  and  proven  tactics,  techniques,  and  procedures  (TTP)  to 
prepare  for  and  conduct  SFA  missions  efficiently.  JCISFA  was  created  to  institutionalize  SFA 
across  DOD  and  serve  as  the  DOD  SFA  Center  of  Excellence.  Find  JCISFA  at  <https://jcisfa. 
jcs.mil/Public/Index.aspx>. 


Support  CAC  in  the  exchange  of  information  by  telling  us  about  your 
successes  so  they  may  be  shared  and  become  Army  successes. 
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